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Introduction 


Charge and Report Organization 

This report is provided to the Court in my capacity as an expert pursuant to Federal Rule of 
Evidence 706 in the case of Parsons v. Ryan, et al. and provides responses to four charges 
contained in two orders issued by the Court. (Doc. 3089 and 3231) The report is organized as 
follows. 

Part I addresses the issue of whether there are “irregularities or errors in the monitoring process 
that produces the compliance numbers for the Stipulation’s 103 health care Performance 
Measures and undermine confidence in their validity” and if “any aspect of the monitoring 
process is unreliable or inaccurate, [provides] written recommendations of remedial measures to 
correct any identified deficiencies.” (Doc. 3089) 

Part IA presents recommendations for the retirement, collapse, or modification of Performance 
Measures (PM). Though not directly anticipated by the Court or by me during the initial stages 
of my work on this review, it became obvious that while some PMs were not reported 
inaccurately, the Court should be aware of an issue with them because, in the best interests of 
this case, one or both Parties, or the health of residents of the Arizona Department of Corrections 
(ADC), the PMs themselves require some remedial action. As explained in more detail in Part 
IA, this Part is responsive, albeit indirectly, to the Court’s charges. 
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Part II presents evidence of “how any failure to successfully perform on PMs poses a significant 
risk of serious harm to patients due to health care delivery that falls below the community 
standard of care.” (Doc. 3194-1 at 4)” to determine “Consistent with the Stipulation and the 
Court’s January 31, 2017 Order, ... whether ‘there is a practice of substantially departing from 
the standard of care.’ (Doc. 3127).” (Doc 3231) 

Part III presents my evaluation of “substantial noncompliance with critical aspects of health care 
delivery including access to prescription medications, diagnostic testing, routine and specialty 
physician consultations, treatment for chronic health care problems, and emergency care (Doc. 
2905),” “the barriers to compliance and propose[d] written recommendations to alleviate them.” 
(Doc. 3089) 

Part IV addresses “whether the PMs by themselves accurately reflect the adequacy of the care 
being provided to prisoners.” (Doc. 3194-1 at 4),” which “analysis likely will assist 
the Court in tailoring the appropriate remedial measures.” (Doc. 3231) In accordance with the 
Court’s indulgence in its order, Part IV will be produced separately, but should be considered an 
integrated part of this (single) report. 

In Parts II and IV, I provide examples of how, in individual cases, health care which is currently 
measured or unmeasured, respectively, by the PMs in this case, pose a significant risk of serious 
harm. My presentation of these examples, however, should not be interpreted as an opinion on 
the overall safety of the systems of care at ADC. I was not charged by the court to evaluate, did 
not design my methodology to, and therefore with rare exception do not offer an opinion on, 
whether, overall, the systems of care in place to deliver health care at the ADC pose a significant 
risk of serious harm to its residents. 

Plaintiffs requested that I organize my report differently, placing Part III and then Part IV at the 
beginning to reflect the importance of substantial non-compliance and whether the PMs 
accurately reflect and measure care. These parts of the report are indeed important. I have 
elected, however, to use the organization that appears here because it more closely matches the 
Court’s order and, more importantly, allows for a more logical presentation and development of 
themes. The order of presentation, therefore, should not be interpreted as reflecting the 
importance of the report parts. 

Methodology 

Between approximately 12/6/18 and the date of this report, I used a number of sources for the 
evidence upon which I base my opinions and recommendations. 

I communicated with (in person, by videoconference, by phone, or by email) the following: 

• Arizona Department of Corrections (ADC) Director 

• More than half of the 21 Arizona Department of Corrections Health Services Monitoring 
Bureau (“Monitoring Bureau”) managerial and program staff at headquarters, including 
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the Board’s Director, Associate Director, the lead Medical, Dental, Mental Health and 
Pharmacy specialists 

• 17 of 21 field-based and headquarters-based Monitoring Board (MB) facility monitors 

• Wardens, associate wardens, lieutenants, sergeants at five facilities 

• Front line correctional officers (CO) 

• Corizon’s manager, assistant manager, and medical director, of the ADC contract 

• Corizon’s chief psychologist and psychiatrist of the ADC contract 

• Corizon’s Clinical Coordinators (staff who shepherd specialist consultation requests 
through the approval process) 

• Several Corizon Facility Health Administrators, Directors of Nursing, Medical Directors 

• Several Corizon front line health care professionals, including providers 1 and nurses 

• Several Corizon front line mental health professionals including psychologists, 
psychiatrists, psychology associates, psychology technicians 

• The office manager of a community specialist who no longer agrees to care for ADC 
residents in his office 

• Two former Corizon employees (nurse, psychologist) who were on the record criticizing 
health care delivery at ADC, in the news media and in court, respectively 

• Assistant Director, Division of Business and Finance, Arizona Health Care Cost 
Containment System (AHCCCS) 

• Program directors of the University of Arizona Telehealth Program 

• Perryville Food Services Manager 

• ADC Food Services Manager 

• Nutritionist Consultant to ADC 

I visited the ADC Monitoring Bureau and ADC complexes at Phoenix, Tucson, Eyman, 

Florence, and Lewis. These facilities were chosen with the input and concurrence of both Parties. 
Plaintiffs suggested that I also visit Perryville because of its different mission and, because of its 
success complying with the PMs, practices there might inform my suggestions for how to 
improve PM performance at other complexes. Taking the suggestion under advisement, after 
completing my visits to the other complexes, I felt that a visit was not warranted. However, after 
reviewing Plaintiffs’ attorneys’ letters to Defendants dated 4/22/19 and 5/6/19 in which they 
alleged serious problems with provision of mental health and perinatal care, triggered by their 
visit to Perryville on April 2-4,1 also visited Perryville. Thus over the course of three trips (8 
days total) between 2/4/19 and 5/16/19,1 visited the six complexes cited above. 

Lastly, I reviewed the following documents: 

• Corizon Staffing Reports 


1 To be consistent with the way the term is used by ADC, I use the term “provider” in this report 
to denote a prescriber, i.e. physician, nurse practitioner, physician assistant, dentist. For the 
former three terms, the individual might be a medical or mental health professional. 


5 




Case 2:12-cv-00601-ROS Document 3379 Filed 10/04/19 Page 6 of 138 


• Relevant portions of the medical record of several hundred patients 

• The medical records and related administrative documents of the 58 patients who died 
between September, 2018 and April, 2019 

• Approximately 75 consecutive advocacy letters sent by the Plaintiffs to the Defendants 

• Approximately 100 of the documents (Orders, Motions, Responses, Testimony 
Transcripts, etc.) filed with the Court in this case over the past five years. I chose these 
documents based on their relevance to my assignment and guidance from both Parties. 

Pursuant to the Court’s Order of 5/30/19 (Doc. 3269) in response to my request, I obtained 
expert assistance in reviewing many mental health records from Dr. Bart Abplanalp, Chief 
Psychologist for the Washington Department of Corrections. I incorporated his input in my 
report. While I generally mention where such input contributed to my opinions and 
recommendations, that attribution is not exhaustive. I directed, oversaw, and, as appropriate, 
verified his work. 

Both Parties were extremely accommodating of my needs for their time, documents, and input. I 
do not feel that my review was impaired in any way due to lack of cooperation or access to 
persons or information. I communicated with both Parties during my research and drafting of my 
report, and sought their input on key issues. My goal was to provide recommendations to the 
Court which were workable for, and acceptable to, both Parties to the extent possible. I also 
circulated the report draft to the Parties and incorporated their input in the final report. 

My only communication with the Court has been about matters of logistics; the Court was not 
aware of my findings prior to submission of this final report nor played any role in shaping my 
opinions or recommendations. 

Finally, I use three terms in this report which deserve explanation. Registered nurses (RN) and 
licensed practical nurses (LPN) are two classifications of nurses (the third classification, nurse 
practitioner, functions as a provider). Both are employed at ADC facilities. While both are 
legitimately referred to as nurses, their training and legal and safe scopes of practice are very 
different. In simple terms, with a few exceptions, LPNs may only collect patient data, report it to 
an RN or provider, and carry out a care plan directed by an RN or provider. RNs may also 
evaluate collected patient data to arrive at an assessment (nursing diagnosis), and design a care 
plan based on that assessment. Thus RNs may conduct a broad range of nursing activities 
independently whereas LPNs may not. I also use the term patient safety. Patient safety is the 
attribute of patient care whereby patients receive the care that was intended and planned without 
error. As used in this report, deficiencies in patient safety pose a significant risk of serious harm. 

Evaluating Performance Using Clinical Judgment 

I propose modifications to current PMs in Parts I and III. Several of these modifications include 
a shift from an objective measure (e.g., counting events or calculating the time between events) 
to a subjective measure wherein a nurse- or a provider-monitor uses his or her clinical judgment 
to assess whether performance is acceptable. For example, in Part IA, I recommend that patients 
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in an infirmary be seen by a provider with a clinically appropriate frequency as determined by 
the treating provider, rather than the current requirement of every 72 hours. Defendants did not 
concur with the subjective component of most of these recommendations. Based on discussions 
with them during my review and their feedback to the draft, their concern is that subjective tests 
are vulnerable to disagreement among medical professionals. In the context of this case, 
Defendants’ concern is understandable. Objective tests should be much easier to adjudicate than 
subjective ones, and even adjudication of objective tests has been challenging to both Parties. 
However, in my opinion, objective tests, by themselves, are simply insufficient to measure 
adequacy of care. Worse, for certain clinical activities, the misapplication of objective tests can 
incentivize the wrong behavior and lead to riskier care. Further, I believe there is a model for 
adjudication which can avert the feared “battle of the experts.” For those - hopefully infrequent 
cases selected for audit - where there is disagreement between the Plaintiffs’ medical/mental 
health experts and the Defendants’ medical/mental health experts as to whether the individual 
case is compliant or not with the PM, the Parties could designate a mutually acceptable 
independent local clinical expert who could make a final determination without resorting to 
judicial review. 

Modification of PMs vs Termination 

For some of the modifications of PMs I recommend in Parts I and III, I have specified that these 
modifications apply going forward. In their response to the draft of this report, Defendants did 
not agree to modifications of a number PMs going forward because they hold that the PM should 
be terminated due to Compliance. I have not reprised the Defendants comment for each of the 
PMs involved, providing this global statement instead. I agree with the Defendants’ point. To be 
clear, where necessary, I made recommendations to modify a PM “going forward” in the event 
that the Court does not terminate the PM. If the Court does terminate the PM, it should consider 
my “going forward” recommendation moot. 
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Part I - Are the reported PM performance levels accurate and if not, recommendations 

(Doc. 3089 at 1) 

The Court asked me to address whether there are “irregularities or errors in the monitoring 
process that produces the compliance numbers for the Stipulation’s 103 health care Performance 
Measures and undermine confidence in their validity” and if “any aspect of the monitoring 
process is unreliable or inaccurate, [provide] written recommendations of remedial measures to 
correct any identified deficiencies.” The Court also instructed that my “analysis will include, but 
is not limited to, a review of the electronic medical records system (eOMIS), whether the 
instructions for evaluating compliance as outlined in the Monitoring Guide are being applied 
correctly, whether the required record review includes a sufficient number of records and a 
proper sampling process, the informal ADC/Corizon challenge process to draft compliance 
numbers, and the documentation of any subsequent modifications to proposed compliance 
numbers.” Of these four components, I address the first 3 throughout this, and other Parts of the 
report, where relevant. I address the fourth component (the informal ADC/Corizon challenge 
process to draft compliance numbers, and the documentation of any subsequent modifications to 
proposed compliance numbers) in the first section below. 

There are 103 PMs in this case, most measured at 10 facilities, yielding approximately 1000 
PM/complex pairs to consider in response to the Court’s request to opine on the accuracy of PM 
performance levels. Reviewing all 1000 pairs was not practical or logical. Instead, I focused my 
attention of PMs based on three drivers: PMs or areas of health services highlighted by the Court 
in Doc. 3089; PMs about which Plaintiffs expressed concern or questioned the validity; PMs 
which, in the course of my work, caught my attention as being, or potentially being, problematic. 

In analyzing the reporting of PM performance levels, it was necessary to understand how the 
audits upon which the performance levels are based are conducted. That understanding was 
based, in part, on the ADC’s Monitoring Guide, revised 2/7/18. However, pursuant to Court 
Orders, the Defendants have made changes to the way they audit various PMs. Thus, my 
understanding of the auditing process was supplemented and amended by multiple interviews 
with multiple monitors who conduct the audits, and their supervisors. Finally, when necessary, I 
verified the audit methodology by test auditing individual cases. Across the span of PMs upon 
which I opine, I test-audited hundreds of individual cases. These cases were chosen purposively 
or somewhat randomly, as appropriate to the question I was trying to answer. 

Issues Affecting Two or More PMs 

“Rebuttals” to Compliance Numbers 

Issue: 

The “informal ADC/Corizon challenge process to draft compliance numbers, and the 
documentation of any subsequent modifications to proposed compliance numbers” 
identified by the Court is ADC’s Rebuttal Process. The Court identified this process as 
one which required expert inquiry. To conduct that inquiry, I used as a framework the 
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concerns expressed about the process by Plaintiffs. (Doc. 2046 at 31) Plaintiffs cited 
seven concerns. The first four are either moot or do not require review. I examined the 
remaining three, which are: 

Fifth, ADC only presents Corizon with the Noncompliant findings for rebuttal; 
there is no parallel process to test the accuracy of findings of Compliance. [Id. at 
42:19- 21] This creates a one-sided process that can only result in improved 
compliance scores. 

Sixth, when ADC headquarters staff accept Corizon’s challenges, they discard the 
challenged files from the sample if the basis for the challenge is that the file was 
not applicable, and they do not randomly select additional applicable files, thus 
changing not only the rate of compliance, but the number of files reviewed. 

Seventh, and most significantly, Ms. Campbell admitted, and the documentation 
proffered by Plaintiffs’ counsel at the hearings showed, that ADC headquarters 
staff have a pattern and practice of going into the CGAR 2 computer system to 
alter entries made by the individual monitors in the CGAR system, to wipe clean 
the monitors’ original findings that had been locked into the CGAR system, and 
replace the previous findings with new information. [3/8/17 Tr. at 22:2-23:5; 
26:15-21; 33:24-34:3, 35:12-36:10] ADC headquarters staff make changes in the 
monitors’ names, without notifying the individual monitor, and without changing 
the original date and time stamp. 

(Doc. 2046 at 31) 

To conduct my review, I met with the ADC official who manages the Rebuttal Process, 
examined his records with him, and examined primary documentation supporting those 
records. In total, I reviewed 20 rebuttals covering a total of about 40-50 patients. I came 
to the following conclusions. As a general matter, I found the process to be organized, 
meticulously documented, and consistently conducted. Where ADC elected to accept a 
rebuttal from Corizon (i.e., reverse a finding of Noncompliant to Compliant for a given 
sampled case), I concurred with those decisions. 

The Plaintiffs’ fifth concern is one of asymmetry: the process is limited to switching 
Noncompliant findings to Compliant, and not the reverse. While this is factually correct, 
it is exactly what a rebuttal process is designed to do and is consistent with the standard 
in the industry. It is unimaginable that the vendor would ever want to rebut Compliant 
findings. However, there are two other processes in place to provide symmetry. First, 


2 CGARs are the monthly reports which contain the cases sampled each month for each PM at 
each complex, whether each case was Compliant or Noncompliant, and the summary score for 
the PM at that complex. 
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Plaintiffs should, can, and do examine CGAR findings. Second, and even more powerful, 
is the way that the PMs are audited by ADC’s monitors. Over the course of my review for 
this report, I had extensive in-person, telephonic, and email communication with almost 
every ADC monitor, questioning them about their monitoring process as well as 
reviewing scores of individual audit decisions with them. It is my firm conclusion that 
ADC monitors conduct their work as fiduciaries for patients incarcerated at ADC. Their 
first and only consideration is whether patients are safe, as measured by the PMs they 
review. If anything, I found their audit decisions, at times, to be less forgiving of the 
vendor’s actions than I might have been myself. Thus while the Rebuttal Process is, 
indeed, asymmetrical, it exists within a larger monitoring process which is adequately 
symmetrical. 

With regard to the Plaintiffs’ sixth concern, that of the rejection of a rebutted case (when 
Corizon successfully argues that the case that was sampled and found Noncompliant was 
not eligible to be audited) resulting in an overall reduced sample size, I found that either 
to not be the case or to not matter. Typically, ADC replaces any rejected case with the 
next case on the randomized list. This is statistically appropriate. However, at those 
specific times when one case more or less in the sample would not change the overall 
compliance level (in either direction), ADC does not replace, re-audit, and recalculate the 
PM. This too is statistically appropriate. 

With regard to Plaintiffs seventh concern regarding how the final version of the CGARs 
reflect rebuttal-driven corrections, they are correct in part and incorrect in part. If ADC 
wishes to make corrections to the CGARs after they have been posted by the Information 
Technology Department, the computer programming will, in fact, only allow them to do 
so by removing the original monitor’s findings and name, replacing it with the new 
findings under the name of the person who manages rebuttals. However, (a) this does not 
change the conclusion for the purposes of the Stipulation, and (b) ADC maintains an 
accurate paper trail of the original monitor’s audit results and the evolution of the 
correction if it were ever necessary to track the history. With regard to involvement of the 
original monitor in the Rebuttal Process, with rare exception, the Rebuttal Process 
manager informs the monitor of any rebuttal and seeks their input. If the monitor concurs 
with the proposed change, typically the change is posted under the monitor’s name. If 
they do not concur, the issue is escalated within ADC for a final opinion. If the monitor is 
not comfortable with a final decision to change a finding from Noncompliant to 
Compliant, the change is posted under the manager’s name. This is the appropriate way 
for it to be handled. Once again, an adequate paper trail is maintained. 

In summary, I found the Rebuttal Process employed by ADC to be fair, statistically 
sound, and well documented. 

Recommendation 1: 

None. 
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Defendants concur 3 . Plaintiffs were not able to concur, providing the following 
explanation: “Defendants have not provided Plaintiffs with copies of the rebuttals for any 
month since August 2017 despite a standing request for them; therefore Plaintiffs cannot 
independently determine whether the audit trail is appropriate.” 


Unit of Analysis 

Issue: 

A concern that Plaintiffs have raised relative to several PMs is that when randomly 
choosing events for a given PM, a given patient might appear in the sample more than 
once (e.g. a patient might have submitted two HNRs 4 during the month, and both HNRs 
might, by chance, have been selected as two of the ten events for that yard). An analysis 
of whether this is the correct way to sample relies on a statistical concept called Unit of 
Analysis (UoA). When one decides to use patients as the UoA, then only one event 
should be sampled per patient (this can be accomplished either by: (a) randomizing all 
events, but if a second event is encountered for a given patient, that event is skipped, or 
(b) by randomizing all patients, and then selecting one event from each patient). If, on the 
other hand, one decides to use events as the UoA, then any event is eligible for sampling, 
even if that results in two events being selected, both of which occurred with the same 
patient. Both methods are used in scientific endeavors; which one is the correct one to use 
depends on the goal of the question and the context in which the events occur. As a 
general rule, if each event within a patient is likely to be independent 5 of other events 
within the same patient, it is reasonable - and in certain situations, even desirable (see 
below) - to use the event as the UoA. 

When monitoring PMs, ADC had been using the event as the UoA and on occasion 
(especially for patients with more rare conditions), a given patient’s events have been 


3 The Parties noted that “concurrence” is a legal term of art and may have implications beyond 
my intent. In written and oral discussions of the recommendations in my report, the Parties have 
variously used terms such as “agree,” “do not dispute,” “concur.” In the absence of another term 
upon which the Parties agreed, I use the term concurrence throughout the report to simply 
indicate that one or both parties found no significant flaws in my analysis and recommendation 
and did not dispute them. 

4 Health Needs Request. These are slips of paper filled out by patients by which they 
communicate health care needs to health care personnel. They drop them in a collection box 
from which they are collected by health care personnel, and reviewed for appropriate action, 
usually a visit with a nurse. 

5 To illustrate using an example, if Patient A submits an HNR on January 1 st and another one on 
January 10 th , and medical staff would not tend to treat those two HNRs in the same manner (e.g. 
giving them higher priority or lower priority) just because they came from Patient A, we would 
say those two HNRs are independent of each other. 
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sampled more than once. Based on my understanding of how health care is generally 
delivered in prisons as well as my specific knowledge of how health care is delivered in 
ADC, for most PMs, I find ADC’s use of the event as the UoA method appropriate for 
most, but not all PMs. This issue was also brought before the Court specifically with 
regard to PMs 94, 95, and 97. Based on testimony from Plaintiffs expert (Doc. 2090 at 
3), the Court ordered ADC to use the patient, and not the event, as the UoA. (Doc. 2185 
at 2) Unfortunately, when the number of patients and events is small, which has happened 
with PMs 94, 95, and 97, using the patient as the UoA, as ordered by the Court, could 
result in a small sample which can yield misleading results. 

Recommendation 2: 

Going forward, ADC should use (or continue to use) the event as the UoA for PMs 
(unless otherwise specified). The Court’s previous order directing ADC to use patient as 
the UoA (Doc. 2185 at 2) should be reversed with regard to PMs 94, 95, and 97. 

The Parties concur. 

Accuracy of Labeling of Psychiatrist Encounters as to Authorship 

Issue: 

Plaintiffs noted an encounter conducted by an RN but mislabeled in eOMIS as being an 
encounter by a psychiatrist. “Labeling” here refers to the way the document is entitled in 
eOMIS. The title of the document allows users to search for particular types of 
documents; it is distinct from the actual signature of the author. The Plaintiffs’ concern is 
that if encounters are mislabeled, this could result in errors in measurement of PMs 81, 

83, 84, 85, 88, 90, 91, and 95, which draw samples from documents entitled as 
psychiatrist, or other mental health provider encounters (but not RNs) to measure whether 
these professionals have provided the specified service. 

To assess this concern, first I interviewed MB staff involved in conducting the audits of 
the relevant PMs. I was told that the auditor reads the actual encounter note, including the 
signature of the author. Therefore, an encounter conducted by, and authored by, an RN, 
would be appreciated for what it is, regardless of the fact that the encounter was 
mislabeled as a psychiatrist encounter. Second, I conducted a test to determine the 
frequency with which documents labeled as psychiatrist encounters are actually be 
authored by a non-psychiatrist. I reviewed 54 documents labeled as psychiatrist 
encounters across 11 patients at six complexes. All 54 documents were correctly labeled. 

I conclude that mislabeling of mental health encounters is not a material concern. 

Recommendation 3: 

None. 

The Parties concur. 
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Measurement Variability 

Issue: 

Over 20 different monitors from the ADC Monitoring Bureau participate in the auditing 
of the 103 medical PMs. For many PMs, auditor (monitor) assignments are based on the 
complex to which the monitor is physically assigned. (This method of assigning monitors 
is a vestige of the days when the ADC used paper patient health records; auditing 
required on-site review of those paper records.) For these PMs, there may as many as 10 
different monitors auditing the PM across the 10 complexes. I discovered considerable 
variability in the way different monitors score some of these PMs. For example, when 
scoring PM 44 ( Inmates returning from an inpatient hospital stay or ER transport with 
discharge recommendations from the hospital shall have the hospital’s treatment 
recommendations reviewed and acted upon by a medical provider within 24 hours. ) if the 
facility provider substitutes a medication recommended by the hospital with another 
similar medication, some monitors will accept this as Compliant while others, 
interpreting the PM more strictly, will not. Except as noted elsewhere in this report, this 
variability tends to result in lower , not higher, compliance scores. Now that ADC uses an 
entirely electronic health record, on-site review of the paper record is no longer 
necessary. 

Recommendation 4: 

In the interest of consistent PM measurement across complexes, ADC should assign 
monitors according to PM, not complex, i.e., for a given PM, a single monitor should 
measure the PM at all complexes. The only exception to this pattern should be for PMs 
which require on-site observations, e.g. PM 24 ( Emergency medical response bags are 
checked daily, inventoried monthly, and contain all required essential items.). 

The Parties concur, and in fact Defendants note that they are already “in the process of 
reassigning monitoring where possible.” 


Processing of Requests for Specialty Services 

Issue: 

Five PMs address the handling of requests for specialty services. These are requests from 
a facility provider to send a patient to an external specialist for a diagnostic service (e.g. 
MRI, biopsy), a consultation, or a therapeutic procedure (e.g. surgery, radiation therapy). 

I discovered a problem with the Source Document 6 used to audit these PMs. The problem 


6 A Source Document is a list of patients or patient events generated on a monthly basis of all 
patients or patient events that correspond to the aspect of care being measured by a PM. Some 
PMs require their own Source Document; some share Source Documents. The Source Document 
list is randomized by complex and by yard within complex, after which the first 10 items on the 
list from each yard are subjected to testing to yield a global score for that PM at that complex. 
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I discovered would not logically affect PM 52, and given that I opine elsewhere that 
failure to successfully perform on PM 49 does not pose a significant risk of serious harm, 
I limit my discussion here to the remaining three PMs for which the problem with the 
current Source Document may render the PMs results inaccurate and for which, therefore, 
I find the reported performance levels unreliable: PM 48 ( Documentation, including the 
reason(s)for the denial, of Utilization Management denials of requests for specialty 
services will be sent to the requesting Provider in writing within fourteen calendar days, 
and placed in the patient's medical record.)', PM 50 (Urgent specialty consultations and 
urgent specialty diagnostic sendees will be scheduled and completed within 30 calendar 
days of the consultation being requested by the provider.)', and PM 51 (Routine specialty 
consultations will be scheduled and completed within 60 calendar days of the 
consultation being requested by the provider.) 

In brief, provider requests for specialty services were processed by Corizon as follows: 

• Providers enter his/her request in eOMIS. 

• A clerk (known as the Clinical Coordinator) at the complex acknowledges 
receiving the request. (At this point, the clerk changes the status of the request to 
“Clinical Coordinator Status.”) 

• The clerk transfers the request manually from eOMIS to another software 
program used by Corizon (“CARES”). When so doing, the clerk changes the 
status of the request in eOMIS to “Referred to UM [Utilization Management] 
Team for Review.” 

• As the request goes through the UM process, the clerk changes the status of the 
request is eOMIS as appropriate (for example, “Need More Information,” “More 
Information Provided,” “Alternative Treatment Recommended,” “Alternative 
Treatment Accepted”). 

I discovered two problems with the Source Documents used for these three PMs. First, 
the Source Documents are drawn from the list within eOMIS of specialty requests that 
are under way or have been resolved/completed. As such, they do not capture requests 
that were cancelled early on without having been processed through the appropriate 
review process conducted by Corizon’s headquarters-based Utilization Management 
(UM) department * * 7 . These uncaptured requests are potentially of great importance because 
these might be requests which were clinically necessary, but cancelled by the vendor for 
unsound reasons, e.g. there was a delay in scheduling the request and the vendor does not 
want to be found Noncompliant with the PM. Second, a request whose status is changed 
from “Urgent” to “Routine” without proper clinical justification will appear on the 
Source Document prepared for PM 51 (which examines management of Routine 


The Source Document for most PMs is drawn from the appropriate database within the electronic 

medical record, eOMIS, by the vendor. 

7 [deleted] 
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requests). This is an error because, in principle, it is still an urgent request and should be 
completed within 30, not 60, days, but if it is audited under PM 51 and is completed 
within 60 days, it will be found Compliant even though it was completed after 30 days . 

It is because of these uncaptured or misclassified requests that I find the reported 
performance levels for PM 48, PM 50, and PM 51 unreliable. The following information 
supports my opinion. 

In a letter to Defendants, dated 5/7/19, Plaintiffs alleged post hoc medical record 
modification by Corizon in the case of Patient 43 8 .1 investigated this allegation by 
reviewing the patient’s medical record and discussing the case and documentation with 
the clerk (“Clinical Coordinator”) who made the record modification and Corizon’s 
Arizona Medical Director. In brief, on 4/10/19, a physician requested urgent 
neurosurgery for the patient. As this was an urgent request, PM 51 requires that the 
surgery be performed by May 10. Due to the surgeon’s schedule, surgery could not be 
scheduled until 5/15/19. After checking with the requesting physician, the clerk 
Coordinator accepted the May 15 surgery date, and changed the urgency of the request 
from “Urgent” to “Routine” so that the vendor would not appear out of compliance with 
the time requirement of PM 50. While, in my opinion, completion of surgery in 35, rather 
than 30, days did not pose a significant clinical risk to the patient, and was clinically 
appropriate given the context of the case, the post hoc modification of the record as well 
as the lack of documentation by the requesting physicians clinically justifying the change, 
calls into question the reliability of historical results that rely on these Source Documents. 

In another case, during a visit with a provider on 1/18/18, Patient 12, who had recently 
undergone treatment for skin cancer (basal cell carcinoma), was noted to have new 
suspicious lesions. On the same day, the provider generated two consults to the 
dermatologist who had been treating the patient, one requesting follow up of a procedure 
the dermatologist had recently performed to remove skin cancer (written as a “Routine” 
request), and another one requesting the dermatologist to evaluate two new skin lesions 
which were increasing in size (written as an “Urgent” request). The clerk cancelled the 
Urgent consult. The only documentation explaining this cancellation was an entry by the 
clerk: “Duplicate, two consult requests for dermatology entered on the same date.” Even 
under the assumption that the requests were duplicates (which they were not), if one of 
two duplicate consultation requests were to be cancelled, good judgment would dictate 
that it be the “Routine” one. Given the patient’s worrisome history of melanoma skin 
cancer (provider visit of 3/30/19) and basal cell skin cancer, I could find no evidence in 
the medical record to justify a “Routine,” rather than “Urgent,” consultation. 

Additional information came from testimony. One of the Corizon clerks testified that 
when Corizon’s UM department replied to the provider’s request asking for more 


8 The legend of patient names have been provided to both parties under separate cover. 
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in formation (“Need More Information”; “NMI”) and obtaining that additional 
information was expected to take a while, her Corizon supervisors instructed her (and 
other clerks) to cancel the consult and then reenter it when the needed information 
became available. Clerks were instructed to also cancel requests when they anticipated 
delays in identifying an available specialist in the community. (Doc. 2876 at 18 and 21) 

Finally, at my request, staff at the Monitoring Bureau conducted a test. They searched for 
all provider requests for specialty care that had been cancelled. Of nine consults 
randomly chosen for review, the Bureau found three which did not appear on the relevant 
Source Documents and therefore would not have been subject to review under PM 48, 
PM 50, or PM 51. That such a high percentage of specialty requests might not be subject 
to PM review is quite concerning. On the positive side, the Source Document used by 
ADC beginning with the January, 2019 CGAR report, did include cancelled consults, 
thus performance levels reported for January, 2019 and subsequent months are reliable. 

The fix I recommend for this issue relies, in part, on the fact that based on my reviews of 
the cases that were included in the audits of these PMs, the reported results for these PMs 
are accurate. Thus there is no need to redo all the previous work, but rather ensure that 
the appropriate cases were sampled and that they were classified correctly to be audited 
in PM 50 (“Urgent”) vs. PM 51 (“Routine”). 

Recommendation 5: 

I recommend retrospectively re-auditing PM 50 and PM 51 for any month prior to 
January, 2019 ADC intends to use as evidence of compliance and which is currently 
Compliant, according to the following protocol. As an overview, the protocol I 
recommend below for fixing the two problems with PM 50 and PM 51 (cancelled 
requests missing from the Source Document, or requests that were rightfully “Urgent” 
being misclassified to the “Routine” Source Document) is a three-step process. The 
protocol reduces the workload of re-auditing by preserving as much of the sample 
originally used to calculate the PM as possible. As an overview, the protocol makes sure 
the samples for PM 50 and PM 51 don’t exclude consults that were cancelled, makes sure 
the sampled cases are correctly classified as Urgent or Routine, and then makes sure that 
the resultant samples are correctly audited. 

Step 1 

The first step corrects any errors in the sample that was chosen for the original audit by 
replacing cases within that sample with missed cases if such missed cases would have 
been randomly selected for audit if the sample of 10 per yard had been drawn from a list 
of requests that included consults which were cancelled at a very early stage of the 
request process. 
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1.1. Generate a list of all Urgent specialty requests that were requested in the 
month under review and a list of all Routine specialty requests that were 
requested in the month under review. 

(To allow enough time to elapse from the date a consult is requested until it 
should have been completed, PM 50 examines consults requested in the month 
prior to the audit month whereas PM 51 examines consults requested two months 
prior to the audit month.) 

1.2. Randomize the two lists. 

1.3. Compare the first 10 cases for each yard each of the two lists to the entire 
original Source Document for that audit. 

1.4. If any of those 10 new cases are missing from the original Source Document, 
move those cases into the original audit sample and “bump” the same number of 
cases from the bottom of the original audit sample. These two newly constituted 
lists of 10 cases (each a combination of old and new cases) are then subject to 
Step 2. 


Step 2 

The second step checks the newly constituted sample for PM 51 (“Routine”) to make sure 
that any cases (old or new) which were inappropriately changed from “Urgent” to 
“Routine” during the UM review process are moved to the PM 50 (“Urgent”) sample. 
After the shift, cases are added to the PM 51 (“Routine”) list, if necessary, to bring it 
back up to 10, and cases are removed from the PM 50 (“Urgent”) list, if necessary, to 
bring it back down to 10. 

2.1. Examine the sample from Step 1 for PM 51 (“Routine”) for the month under 
review. Review the provider’s specialty request to determine whether, when first 
generated, the provider requested the consultation as Routine or Urgent. 

2.2. If the provider originally requested the consultation as Urgent, then the 
monitor, in collaboration with the Monitoring Bureau physician, will determine if 
the provider documented a prospective order (i.e., written before the consult 
urgency was changed) to change the urgency of the request from Urgent to 
Routine and provided sufficient information to support the clinical 
appropriateness of the change. 

2.3. If the appropriateness of the change in urgency cannot be supported, move 
the case from PM 51 (“Routine”) to PM 50 (“Urgent”) for the same month’s 
audit 9 , where it will “bump” the last case. 


9 The Source Documents for PM 50 and PM 51 draw from two different months, for the reason 
explained earlier. Technically, cases moved between these two PMs therefore belong in different 
months. However, as long as the same method is used consistently month after month during this 
re-audit, it is simpler, and still statically acceptable, to keep them in the same audit month, as this 
method instructs. 
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2.4. Replace any cases moved out of PM 51 (“Routine”) with the next unused 
case from the new randomized list generated for PM 51 (“Routine”) in Step 1. 


Step 3 

In the third step, the auditor audits any new or reclassified cases now in the two samples 
as a result of Steps 1 and 2, and recalculates the final compliance score. 

3.1. Audit all new cases in the two reconstituted samples for PM 50 (“Urgent”) 
and PM 51 (“Routine”) to determine if they meet the time requirement (30 days or 
60 days, respectively). 

If a request in a case was cancelled, audit this case in collaboration with the 
Monitoring Bureau physician: if a clinically reasonable justification for 
cancellation was prospectively documented by the patient’s provider or is 
otherwise clinically appropriate (e.g. identical duplicate request), the case is found 
Compliant. 

If a consult was cancelled or rescheduled by the consultant's office AND it was 
re-scheduled to occur within the timeframe specified by the consultant, the case 
should be considered Compliant, even if that date is beyond the 30- or 60-day 
timeframe. 

If a consult took place beyond the 30- or 60-day timeframe due to complex 
operational issues such as a unit lockdown, the case should be considered Non- 
compliant. 

3.2 Recalculate the new compliance score for each of the two samples. 

The Parties concur. 

Recommendation 6: 

Going forward, ADC should measure PM 50 and PM 51 using the Source Document 
described in Step 1.1 of Recommendation 5, and the criteria described in Step 3.1 of 
Recommendation 5 above. 

The Parties concur. 

Recommendation 7: 

I recommend retrospectively re-auditing PM 48 for any month ADC intends to use as 
evidence of compliance and which is currently Compliant, according to the following 
protocol. As an overview, the protocol I recommend below for fixing the problem with 
PM 48 (cancelled requests missing from the Source Document) mirrors Steps 1 and 3 of 
Recommendation 5 above. 


1. Combine the two lists generated in Step 1.1 in Recommendation 5 (i.e. a list 
that includes all Urgent and Routing that were requested in the month under 
review) 
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2. Randomize the list. 

3. Compare the first 10 cases for each yard each of the list to the entire original 
Source Document for that audit. 

4. If any of those 10 new cases are missing from the original Source Document, 
move those cases into the original audit sample and “bump” the same number of 
cases from the bottom of the original audit sample. This newly constituted list of 
10 cases (a combination of old and new cases) is then subject to the next step 

5. Audit all new cases in the reconstituted sample to determine if they meet the 
time requirement of PM 48. 

6. Recalculate the new compliance score for each of the two samples. 

The Parties concur. 

Access to Care Following a Request from a Patient 

Six PMs measure the degree to which episodic requests from patients for health care 
(mental health, medical, or dental) are handled in a timely manner: PM 36 (A LPN or RN 
will screen HNRs within 24 hours of receipt.)', PM 37 (Sick call inmates will be seen by 
an RN within 24 hours after an HNR is received (or immediately if identified with an 
emergent need, or on the same day if identified as having an urgent need ) 10 ; PM 39 
(.Routine provider refermls will be addressed by a Medical Provider and referrals 
requiring a scheduled provider appointment will be seen within 14 calendar days of the 
referral.); PM 40 ( Urgent provider referrals are seen by a Medical Provider within 24 
hours of the referral.); PM 41 (. Emergent provider referrals are seen immediately by a 
Medical Provider.); and PM 98 (Mental health HNRs shall be responded to within the 
timeframes set forth in the Mental Health Technical Manual (MHTM) (rev. 4/18/14), 
Chapter 2, Section 5.O.). These six PMs share common Source Documents and methods 
of measurement. I found problems with the choice of Source Documents as well as the 
way in which the Source Documents are interpreted, which may render the PMs 
inaccurate; I therefore find the reported performance levels unreliable. 

Issue 1: 

PM 36 and PM 98 (of the five categories of health care requests examined under PM 98, 
this discussion only applies to those HNRs which contain an Emergent request, or an 
Urgent Non-Medication-Related request) measure time intervals based on the date the 
HNR was “received.” To determine the date received, monitors use the date stamp placed 
on the HNR by Corizon staff. Based on my review, there are, at times, large enough gaps 

10 PM 37 was poorly written. When read in conjunction with PM 36, it is clear that the intent was 
for PM 36 and PM 37 to measure two consecutive events: an HNR is received and triaged on 
paper, and then the patient is seen by a nurse. Instead, PM 37 was written as if both events are 
concurrent. Fortunately both Parties have interpreted PM 37 as consecutive to PM 36, and that is 
the way it has been measured. Thus PM 37 is interpreted as “ Sick call inmates will be seen by an 
RN within 24 hours after an HNR is triaged (or immediately if identified with an emergent need, 
or on the same day if identified as having an urgent need). 
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between the date the HNR was written by the patient and the date it was stamped by 
Corizon staff, to question the accuracy of the assumption that the stamped date is, in fact, 
the date received. In other words, the performance levels for PM 36 and PM 98 may be 
misstated. 

I therefore attempted to determine whether the stamped date on an HNR accurately 
reflects the date received, and, if not, to what extent such inaccuracy might cause 
performance levels for PM 36 and PM 98 to be misstated. To determine the expected 
practice with regard to collecting HNRs from the boxes into which patients place them, I 
referred to ADC Department Order 1101. The Order states: “Health Services staff shall 
collect Health Needs Request forms from drop boxes daily, as designed [sic] by the 
Contract Facility Health Administrator.” Thus health care staff should have constructive 
receipt of HNRs the day they are placed in the designated boxes. To determine the date 
they are placed in the boxes, a reasonable indicator is the date written by the patient on 
the HNR. To determine the delay between the date health care staff have constructive 
receipt of HNRs and the date stamped on the HNR, I used the Nurse HNR Log for 
December 2018 at one facility (Eyman, chosen because it was available to me pursuant to 
an earlier request on a different matter). The Nurse HNR Log is a log maintained by 
Corizon nurses to record HNRs they handle. For each HNR, it shows the date the HNR 
was written by the patient and the date it was stamped “received” by medical staff. The 
Nurse HNR Log yielded the following data: 


Gap between the date written 
by the patient on the HNR and 


the date stamped as “received” 

# HNRs 

% 

-1 to -3 days (i.e. patient post- 

21 

1% 

dated the HNR) 



0 days 

1,392 

50% 

1 day pre-dated 

854 

31% 

2 to 30 days pre-dated 

350 

13% 

not recorded on log 

159 

6% 

Total 

2,776 

100% 


These data lead to two conclusions. First, the stamped date on an HNR does not always 
accurately reflect the date an HNR was received. Second, the inaccuracy occurs 
frequently enough (13% of the time, at a minimum 11 ) that it could materially affect the 

11 13% is a minimal level because in my calculation I considered all one-day gaps as accurate (an 
assumption which paints the HNR collection system in the most favorable light). Indeed, 
depending on what time health care staff collect HNRs from the boxes, it is very possible that a 
patient places an HNR in the box after the collection time, and it is not collected until the 
following day. However, if any of those one-day gaps were true delays in collection of HNRs, 
the 13% figure would be higher. 
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calculated performance level for PM 36 or PM 98. In arriving at these conclusions I 
considered the possibility that patients simply write the wrong date on their HNRs. If this 
were happening, I would expect them to make errors in both directions (i.e. pre-dating 
and post-dating) with equal frequency and equal degree (# of days off). However, the 
data clearly shows that that is not the case (see table above): the patient’s written date 
was later than the stamped date (post-dated) only 1% of the time (and then only off by 
one to three days), whereas the patient’s written date was earlier than the stamped date 
13% of the time (and off by much more: 2 to 30 days). 

Finally, I considered the possibility that patients intentionally misstate the date they write 
on the HNR to make the health care staff appear incompetent. While I believe this does 
happen, based on my experience and the content of the HNRs I reviewed at ADC, I 
believe this happens so infrequently as to not materially affect my conclusion that 
performance levels for PM 36 and PM 98 may be misstated. 

It should be noted that if the Court orders re-establishment of the Open Clinic model for 
patient access to episodic care (see Recommendation 58, Part III), this issue will become 
moot. 

Recommendation 8: 

I recommend that PM 36 and PM 98 (of the five categories of health care requests 
examined under PM 98, this recommendation only applies to those HNRs which contain 
an Emergent request, or an Urgent Non-Medication-Related request) be re-audited 
retrospectively for complexes/months reported as Compliant upon which ADC intends to 
rely for termination of the PM, using the date written by the patient, not the stamped date, 
as the date received. To allow for the fact that patients may place an HNR in the box after 
the last pick-up of the day: handling of the HNR should be found Compliant for PM 36 if 
the date of triage is two days or less later than the date written by the patient; handling of 
the HNR should be found Compliant for Urgent Non-Medication requests of PM 98 if the 
date the patient is seen by a nurse or MH staff is two days or less than the date written by 
the patient; for Emergent requests under PM 98, handling should be found Compliant if 
the date the patient is seen is one day or less later than the date written by the patient. 
Patient dates which are exactly one month or one year earlier (or later) should be found 
Compliant as these usually represent a patient error. If a patient request for care is made 
electronically (e.g. kiosk or tablet) or in person (e.g. the patient is brought to the clinic by 
custody staff or presents in person to an Open Clinic), the 24 hour time limit imposed by 
PM 36 and PM 98 begins at the actual time the electronic request is made or the time the 
patient is logged in at the clinic. 

The Plaintiffs concur. Defendants note that HNRs are currently given directly to health 
care staff for patients in living units with restricted movement (maximum custody, mental 
health watch pods, IPC, and CDU), and that that practice would continue, even if ADC 
re-implemented the Open Clinic model (see my Recommendation 58 to re-implement 
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Open Clinics). Because of concerns they have with the accuracy of the date written by 
patients in these living units, the Defendants want the date of receipt for these HNRs to 
continue to be the stamped date for this subset of patients. While some of the patients in 
these units may be a little more challenged than other patients in regard to knowing the 
correct date, my review shows strong evidence that delays in processing HNRs is the 
more likely explanation for apparent delays in care. Thus I have not modified this 
recommendation. Further, because medical staff receive these HNRs personally from the 
patient, they have ample opportunity to note any date error in real time, and ask the 
patient if they will correct it. Defendants also “do not agree that past results are unreliable 
and should be re-audited” but did not offer any specific methodological flaw in my 
analysis. 

Recommendation 9: 

ADC should be allowed, at its discretion, to also recalculate PM 36 and the applicable 
HNRs in PM 98 according to the methodology in the previous Recommendation, for any 
previously reported Noncompliant complexes/months. 12 

The Parties concur, except that for patients in living units with restricted movement, the 
Defendants wish to continue to use the stamped date as the date of receipt. (The 
Defendants’ reasoning and my response are the same as in Recommendation 8.) 
Defendants note that “If a re-audit is done, then compliant facilities’ PMs should be 
terminated.” Defendants also “do not agree that past results are unreliable and should be 
re-audited” but did not offer any specific methodological flaw in my analysis. 


Recommendation 10: 

Going forward, PM 36 and the applicable HNRs in PM 98 should be calculated using the 
method above. However, if and when ADC operates Open Clinics again (see 
Recommendation 58 in Part III), the date and time of receipt of an HNR should be based 
on the date and time written on the clinic sign-in log by the CO (or, if ADC places date 


12 This is the first of several recommendations, all framed as “allowing ADC to re-audit at its 
discretion.” The concept behind all of them is as follows. My analysis led me to believe that 
while the relevant PM was not measured correctly, proper measurement might reveal that 
Defendants performed better than they originally reported. So, in contrast to recommendations in 
which I recommend that Defendants should re-audit months in which a complex was Compliant 
- with the possibility that the results will convert from Compliant to Noncompliant - in this 
group of recommendations, I recommend that Defendants may, if they so desire re-audit months 
in which a complex was Noncompliant - with the possibility that the results will convert from 
Noncompliant to Compliant; if this were to happen, the Defendants would therefore be allowed 
to amend results already filed with the Court. For some PMs it was not clear whether measuring 
the PMs using the proper methodology would result in scores deteriorating or improving, in 
which case I made a pair of recommendations (as is the case here), one requiring re-audit of 
Complaint results, and one allowing re-audit of Noncompliant results. 
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stampers at the CO’s check-in desk for use by the patient when they arrive, the date and 
time stamped by the stamper) when a patient first presents to the Open Clinic to be 
seen 13 , regardless of when - or if - the patient is seen by medical staff. 

The Parties concur, except that for patients in living units with restricted movement, the 
Defendants wish to continue to use the stamped date as the date of receipt. (The 
Defendants’ reasoning and my response are the same as in Recommendation 8.) 

Issue 2: 

The six PMs which are the topic of this section (36, 37, 39, 40, 41, and 98) use 
incomplete Source Documents. For PM 36 and PM 37, ADC uses as the Source 
Document the Nursing HNR Log which, purportedly, captures all HNRs (medical, 
dental, mental health) the nurses handle during the month. For PM 39, PM 40, and PM 
41, ADC uses as the Source Document the Nursing Line Log, which, purportedly, 
captures the name of all patients (medical, dental, mental health) seen by the nurses 
during the month. For PM 98, ADC uses the Mental Health HNR Log which, 
purportedly, captures all mental health-related HNRs the nurses and mental health staff 
handle during the month. These three logs are manual logs, meaning that nurses manually 
enter the visits or HNRs into an Excel spreadsheet. The logs are also free-standing, 
meaning that the spreadsheets are maintained separately from eOMIS. Thus the 
completeness of these logs depends wholly on nurses remembering to, and choosing to, 
record an event. 

I therefore attempted to determine whether the three logs used as Source Documents are 
complete, and, if not, whether the incompleteness might cause performance levels for the 
six PMs to be misstated. To make this determination, I identified two other sources of 
information residing in eOMIS. The first is the record nurses make of encounters with 
patients (HNRs per eOMIS Encounters). The second is the scanned copy of paper 
HNRs submitted by patients (HNRs per eOMIS Scanned HNRs). I requested these 
documents from ADC for Eyman complex for the month of December, 2018 14 . In my 


13 In their comments to the draft of this report, Defendants note that during an audit there may be 
difficulty in distinguishing between HNRs submitted by (a minority of) patients in living units 
with restricted movement (maximum custody, mental health watch, IPC, CDU), where the date 
written by the patient is the basis for calculating timeliness of access to care, versus HNRs 
addressed through the Open Clinic, if the Open Clinic model is re-implemented, where the date 
written by the CO upon arrival is the basis. Given the fact that HNRs include the patient’s 
current location and therefore easily identify patient who are in one of the restricted movement 
living units, there should be little difficulty in distinguishing the two groups of HNRs. 

14 I chose this complex/month for convenience as I already had some of the related documents 
for this complex/month. The actual request for documents I made was as follows: 1. For the 
month of December, 2018, for Eyman complex, a list of all events where: the field called “Type” 
(in the tab called “Encounters” in eOMIS) = “Nurse - Chart Review” OR “Nurse - Chart Note” 
OR “Nurse - Sick Call - Unscheduled” OR “Nurse - Sick Call - Scheduled.” For each event 
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opinion, the union of these two data sources from eOMIS provides the most complete list 
(my “gold standard”) of all HNRs handled, and patients seen, by nurses, for two reasons. 
First, it is much less likely that a nurse would forget to, or choose not to, record an event 
in eOMIS. Second, other office staff help with the scanning and electronic filing of the 
scanned copy of paper HNRs, so the process is not always dependent on a single person. 

I analyzed the PM results by attempting to cross-reference HNRs across all five data 
sources (i.e. HNRs per eOMIS Encounters; HNRs per eOMIS Scanned HNRs; 
Nursing HNR Log (the Source Document ADC uses for PMs 36 and PM 37); Nursing 
Line Log (the Source Document ADC uses for PM 39, PM 40, and PM 41); and Mental 
Health HNR Log (the Source Document ADC uses for PM 98)). My sample included 80 
events identified in one or both of the eOMIS sources. The methodology for construction 
of this sample is described in the footnote 15 . Because of the paucity of mental health 
events in this sample, limiting my ability to draw conclusions about PM 98,1 analyzed a 
second sample limited to a cross-reference of HNRs between HNRs per eOMIS 
Scanned HNRs and the Mental Health HNR Log (the Source Document ADC uses for 


include: patient name, ADC#, “Date,” “Type.” 2. For the month of December, 2018, for Eyman 
complex, a list of all events where: the field called “Document Type” (in the tab called “Scanned 
Documents/Photos” in eOMIS) = “Health Needs Requests.” For each event include: patient 
name, ADC#, “Date Scanned,” “Title.” 

15 The details of my sampling methodology are as follows. For the first sample, I used the first 
(consecutive) 50 HNRs from the combined two eOMIS sources (HNRs per eOMIS Encounters 
and HNRs per eOMIS Scanned HNRs; combined, the “gold standard”), for which the HNR was 
written on or after 12/1/18. To achieve some variation in time of the month and day of the week, 
I supplemented the sample with the first (consecutive) 14, 8, and 8 HNRs from the eOMIS 
sources for 12/14/18 (Friday), 12/15/18 (Saturday), and 12/17/18 (Monday), respectively. For 
each of these HNRs identified in the “gold standard” I attempted to identify the corresponding 
event in the three other Source Documents used by ADC for PM 36, PM 36, PM 39, PM 40, and 
PM 41.1 excluded from my analyses any event which was not applicable. For example, if a 
patient was not referred by the nurse to a provider after the nursing encounter, the event was 
excluded for my analyses of PM 39, 40, and 41. 
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PM 98). The methodology for construction of this sample is described in the footnote 16 .1 
have provided the raw data (including patient names) used for both of my analyses to 
both Parties under separate cover (entitled “Sample 1” and “Sample 2”). 

My first analysis produced the following results. For PM 36, the compliance level was 
90% (37/41) according to source documents ADC currently uses, but was only 86% 
(64/74) according to eOMIS source documents (“gold standard”). For PM 37, the 
compliance level was 98% (39/40) according to source documents ADC currently uses, 
but was only 88% (63/72) according to eOMIS source documents (“gold standard”). For 
PM 39, the compliance level was 51% (19/37) according to source documents ADC 
currently uses, but was only 49% (26/53) according to eOMIS source documents. For PM 

40, there was only a single urgent referral among the 80 events I identified, and for PM 

41, there were no emergent referrals among the 80 events I identified; I therefore did not 
conduct any further analysis of these two PMs. 

For PM 98, the compliance level was 100% (2/2) according to source documents ADC 
currently uses, but was only 40% (2/5) according to eOMIS source documents (“gold 
standard”). As noted above, due to this paucity of cases identified for PM 98,1 conducted 
the second analysis which produced the following results. The compliance level was 78% 
(11/14) according to source documents ADC currently uses, but was higher - 80% 

(16/20) - according to eOMIS source documents (“gold standard”). 

In neither analysis did any event appear only in the Source Documents currently used by 
ADC and not in at least one of the two eOMIS-based documents; in other words, the 
appropriateness of my use of the two eOMIS-based documents as the most complete 
source of information (“gold standard”) was validated. 


16 The details of my sampling methodology for the second sample are as follows. I used the first 
(consecutive) 20 HNRs from HNRs per eOMIS Scanned HNRs for which the date of the 
document, according to eOMIS, was on or after 12/1/18 and for which the title of the scanned 
document in eOMIS indicated a mental health concern (e.g. “speak to psych”) and attempted to 
identify the corresponding event in the Mental Health HNR Log. It should be noted that this 
method likely did not capture all mental health-related events because: (1) I only used the HNRs 
per eOMIS Scanned HNRs, and not the entire “gold standard” as the source. I did this for 
convenience, because there are a great many HNRs on the HNRs per eOMIS Encounters list, 
most of which are not mental health-related. The only way to figure out if these HNRs are mental 
health-related is to open each encounter. Given the vast outnumbering of mental health HNRs by 
non-mental health ones, looking for the mental health-related HNRs using the gold standard 
(which includes HNRs per eOMIS Encounters) would have been exceedingly time consuming 
(“looking for the needle in the haystack”). (2) I did not include any of the great many HNRs 
where the title of the scanned document in eOMIS simply included a generic need, such as 
“medication” or “medication renewal,” some of which probably related to mental health 
medications. I do not believe either of these two procedures significantly affected my 
conclusions. 
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These data lead to three conclusions. First, the Source Documents ADC currently uses for 
measuring performance on PM 36, PM 37, and PM 39 are not complete and such 
incompleteness might cause performance levels for the three PMs to be misstated (over¬ 
stated). Second, the Source Documents ADC currently uses for measuring PM 98 are also 
not complete. However, the data are ambiguous about the extent and direction of any 
misstatement. Third, due to the paucity of events in my sample, and the impracticality of 
using a different methodology, I was not able to draw any conclusion about the 
completeness of the Source Documents ADC uses to measure PM 40 and 41 and 
therefore unable to draw any conclusions as the accuracy of performance levels for these 
two PMs. 

Recommendation 11: 

I recommend that PM 36, PM 37, and PM 39 be retrospectively re-audited for all 
previous complexes/months which are reported as Compliant upon which ADC intends to 
rely for termination of the PM, using the combined Source Document comprised of the 
union of HNRs per eOMIS Encounters (all events in eOMIS where the field called 
“Type” in the tab called “Encounters” = “Nurse - Chart Review” OR “Nurse - Chart 
Note” or “Nurse - Sick Call - Unscheduled” OR “Nurse - Sick Call - Scheduled”) and 
HNRs per eOMIS Scanned HNRs (all events in eOMIS where the field called 
“Document Type” in the tab called “Scanned Documents/Photos” = “Health Needs 
Requests.”). 

In conducting this re-audit, the Defendants should use the same “bumping” method 
described above in Recommendation 5, Step 1. Briefly, “bumping” will shorten the re¬ 
audit by allowing auditors to use as much of the original sample as possible, correcting 
any errors in that sample by replacing cases within that sample with missed cases if such 
missed cases would have been randomly selected for audit if the sample of 10 per yard 
had been drawn from the more appropriate Source Document I have described here. 


Plaintiffs concur. Defendants “believe the creation of the same document will take away 
time for patient care. On average, there are 16,269 HNRs each month system wide. To 
create the same source document would require a review of each HNR to ensure there are 
no duplicates and that they are in fact an HNR for the issue described.” The Defendants 
are correct in expecting that there will be duplicates in the new Source Document. 
However, checking for duplicates will be much easier than they foresee. Duplicates will 
only need to be sought among the 10 cases randomly sampled for each yard. And among 
these 10 cases, the auditor will only need to review an HNR for possible duplication in 
the unlikely event that the same patient name appears on the list twice. I do not expect 
this to be burdensome. Defendants also “do not agree that past results are unreliable and 
should be re-audited” but did not offer any specific methodological flaw in my analysis. 
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Recommendation 12: 

Going forward, ADC should use the above methodology for calculating PM 36, PM 37, 
and PM 39. 

The Parties concur. 

Recommendation 13: 

I recommend that PM 98 be re-audited retrospectively for previous complexes/months 
reported as Compliant upon which ADC intends to rely for termination of the PM, using 
the combined Source Document comprised of the union of HNRs per eOMIS 
Encounters (all events in eOMIS where the field called “Type” in the tab called 
“Encounters” = “Nurse - Chart Review” OR “Nurse - Chart Note” OR “Nurse - Sick 
Call - Unscheduled” OR “Nurse - Sick Call - Scheduled” AND ALSO where the field 
called “Embedded Form” = “NET-Mental Health Compliant”) and HNRs per eOMIS 
Scanned HNRs (all events in eOMIS where the field called “Document Type” in the tab 
called “Scanned Documents/Photos” = “Health Needs Requests” and “Name of 
Encounter/Document” contains any of the following terms: mental; MH; psych*; stress; 
depress*;cope; coping 17 ). However, due to the ambiguous results of my analysis (i.e. that 
it is not clear that use of a more complete data source would result in lower compliance 
levels), the first pass of the re-audit should be limited to every third month. (If that third 
month was previously found Noncompliant, the next closest Compliant month should be 
chosen. Under the “third-month” method, the oldest (first) month to audit is the month 
that is 24 months prior to the month in which this recommendation is approved by the 
Court, followed by the fourth month, and so forth.). If all months for a complex remain 
Compliant, no further re-auditing of the complex is necessary. If, however, this re-audit 
results in any single month level moving from Compliant to Noncompliant, all relevant 
months should be recalculated for that complex. By “relevant” I mean any month that 
ADC plans to use as evidence of Substantial Compliance with the Stipulation agreement. 

In conducting this re-audit, the Defendants should use the same “bumping” method 
described above in Recommendation 5, Step 1. Briefly, “bumping” will shorten the re¬ 
audit by allowing auditors to use as much of the original sample as possible, correcting 
any errors in that sample by replacing cases within that sample with missed cases if such 
missed cases would have been randomly selected for audit if the sample of 10 per yard 
had been drawn from the more appropriate Source Document I have described here. 


17 The restriction I placed on this last parameter was added to address a concern of Defendants 
that many scanned HNRs are titled by record-keeping staff with non-specific titles, such as 
“medication concern” which could be mental health-related (and thus relevant to PM 98) or not. 
Auditors would need to open each of these files to see if they should be included in the audit or 
not, which is time consuming. I conducted a test and discovered that more than a quarter of all 
HNRs have such non-specific titles. Thus the number of such HNRs is non-negligible, making 
auditing time-consuming. I believe the restriction, while imperfect, is not likely to significantly 
impact the accuracy of the proposed measure. The Plaintiffs concur. 
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The Plaintiffs concur. Defendants “do not agree that past results are unreliable and should 
be re-audited” but did not offer any specific methodological flaw in my analysis. 

Recommendation 14: 

Going forward, ADC should use the above methodology for calculating PM 98. 

The Parties concur. 

Recommendation 15: 

Because recalculation of PM 98 using the above methodology could reasonably also 
result in improved performance levels, ADC should be allowed, at its discretion, to 
retrospectively re-audit PM 98 for any complexes/months reported as Non Compliant. 

The Plaintiffs concur. Defendants “do not agree that past results are unreliable and should 
be re-audited” but did not offer any specific methodological flaw in my analysis. 
Defendants also note that “Should a re-audit result in compliant findings, this measure 
should terminate.” 

Were Mental Health Patients “Seen”? 

Issue: 

Mental health PMs 73, 74, 76, 78, 80-91, 94, and 95 share a common construct: that 
certain patients need a visit with a mental health professional at some regular interval. 

The term “seen” (or, in the case of PM 78, “face-to-face encounter”) is used: for example, 
PM 73 states “All MH-3 minor prisoners shall be seen by a licensed mental health 
clinician 18 a minimum of every 30 days.” The term “seen” is defined as an 

Interaction between a patient and a Medical Provider, Mental Health Provider or 
Mental Health Clinician that involves a treatment and/or exchange of information 
in a confidential setting. With respect to Mental Health staff, means an encounter 
that takes place in a confidential setting outside the prisoner’s cell, unless the 
prisoner refuses to exit his or her cell for the encounter. (Doc. 1185-1 at 5) 

In earlier Court proceedings the term “seen” was discussed at length in response to 
Plaintiffs’ concern that group therapy or individual visits conducted at cell-front were 
insufficient to satisfy the PMs’ requirement of being seen. The Court ruled that for the 
purposes of these PMs, cell-front visits (unless the patient refuses to exit his or her cell) 
and group therapy do not count as “seen.” This aspect of disagreement about “seen” 
appears to have been resolved. However, toward the end of my review, Plaintiffs also 
raised concerns about very short mental health visits (some as short as 5, 3, or 2 minutes) 


18 The Stipulation defines a mental health clinician as a psychologist or psychology associate. A 
psychologist is a Ph.D.-trained person; a psychology associate is a masters-trained person. 
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at Perryville, Eyman, Florence, Lewis, Yuma, and Phoenix, and asserted that such visits 
may also not satisfy the requirements of these same PMs. The PMs do not contain any 
specific requirements regarding the length of mental health visits (or any other visit type). 

Pursuant to this concern, I reviewed this issue with the assistance of Dr. Abplanalp, and 
with the intention of providing the Court with recommendations. My review led me to 
believe that this is as much a legal issue as it is a clinical one. I therefore do not offer a 
recommendation, but do provide the Court with my analysis for the issues within my ken. 
My analysis is comprised of four parts: clinical (sections I and II), operational (section 
III) verbal or legal (sections IV and V) and statistical (sections VI and VII). 

For at least part of this analysis I address PMs 73, 74, 76, 78, and 80-90 separately from 
PMs 91, 94, and 95, because the latter group deals with management of patients during or 
after placement on watch (due to acute psychotic or suicidal states), whereas the former 
group deals with patients in non-acute chronic care. 

I. (Clinical, Watch-Related) 

With regard to the watch-related PMs, from among the approximately 50 cases cited by 
Plaintiffs in advocacy letters to Defendants stemming from concern regarding short visits 
at Perryville, Eyman, and Florence, I reviewed 18 cases 19 where the short visits in 
question were during or after a watch. In most of the cases, when viewed in the context of 
the patient’s overall care for the acute problem, the fact that the visit was short was not, 
in and of itself, problematic. This is so because very often during watch, the purpose of 
the visit is rather narrow: to determine if the patient has had any significant change since 
the previous visits. This can be true even for a visit during which the mental health 
professional decides that a patient can be advanced to a less intense level of watch. For 
example, if a patient has been determined to be improved and stable over several days of 
constant watch and can appropriately be advanced to 10-minute watches if the 
improvement has been sustained, a short visit can conceivably be adequate for 
determining that sustained improvement. Short visits may also be appropriate when the 
visit is conducted at cell-front 20 (i.e., the patient declines to be taken to an examination 
room) because this is a non-confidential setting and in-depth conversations either risk 
violating the patient’s right to privacy, or engender stilted or less-than-frank information 


19 I used the first few patients listed in each section of these three Plaintiffs’ letters. While 
typically a more random method is preferred when sampling, given that the list from which I was 
working was, itself, not random, selecting randomly from this list would add little scientific 
rigor. I did not, however, make any conscious effort to select cases with any particular 
characteristic or outcome. Also, as explained in a later footnote, I did not include cases from the 
Plaintiffs’ other three Advocacy Letters because I did not come across those letters in my files 
until after the analysis was completed. Given the nature of the issue and cases cited by Plaintiffs, 
I do not believe this exclusion materially affects my conclusions. 

20 Cell-front visits during watch are, unfortunately, very common at ADC. I address this 
elsewhere. 
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from the patient. Finally, longer visits may be contraindicated in patients on watch 
because if they are agitated and uncooperative - a not-uncommon situation in this acute 
setting - pressuring the patient to participate in a longer engagement may cause their 
agitation to escalate. 

II. (Clinical, Non-Watch-Related) 

With regard to the non-watch-related PMs, the Stipulation itself has created an aberrancy 
in mental health care (see further discussion in Part IA of this report). The good 
intentions of these PMs to set a minimal visit frequency in a variety of situations, has had 
the unintended consequence of forcing visits to be scheduled, even when they are not 
needed. Indeed, in the typical clinical setting, the frequency of mental health visits is 
determined on a case-by-case basis that flows from the clinician’s clinical assessment, 
informed by input from the patient. A number of mental health professionals at ADC 
informed me of unnecessary visits that they would not have scheduled but for the 
requirements of the Stipulation. It is not surprising - nor inappropriate - if those visits are 
exceedingly short. 

III. (Operational) 

Setting a minimally acceptable length of time for visits would likely cause an negative 
unintended consequence. For situations when a short visit is clinically appropriate, 
professionals would feel obligated to lengthen the visit to satisfy the PMs. At best this 
would unnecessarily waste resources; at worst it could agitate a patient who wanted to be 
left alone. 

IV. (Verbal or Legal) 

Analysis of the accuracy of reported performance levels for both groups of PMs must 
also take into account a critical issue: what did the Parties mean by “seen” when the PMs 
were created? While this is, in most part, a legal issue, it has a clinical-monitoring 
component. All of the 103 PMs in this case measure whether or not a task was completed 
or completed on time (an objective test). Not a single PM (with the possible exception of 
PM 25 {A first responder trained in Basic Life Support responds and adequately provides 
care within three minutes of an emergency .)) measures the clinical appropriateness of 
care (a subjective test). While I believe that this is an unfortunate and glaring deficiency 
in the PMs, based on my discussion with the Parties and review of documents filed in this 
case, it also is evident that this was intentional 21 . Thus to determine whether a case is 


21 Plaintiffs disagree with my characterization, noting that they “incorporated a subjective 
analysis in the protocol of many of the PMs, but ADC has focused solely on timeframes. To the 
extent there has been any substantive review, i.e. “act upon,” it has been at the behest of the court 
after Plaintiffs brought it to the court’s attention.” They also note “As seen in the protocols 
attached to the stipulation, for many of the PMs, there was the intent that the appropriateness of 
the action(s) would be assessed.” Defendants maintain that “‘Seen’ does not require a subjective 
review of the appropriateness of the contact. There is either treatment provided or an exchange 
of information without a determination of the quality of the contact. This has not been raised 
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compliant by conducting a subjective analysis of whether the amount of time spent 
during the visit was clinically appropriate, seems inconsistent with the Parties’ original 
intent. 

V. (Verbal or Legal) 

Notwithstanding my conclusion in section El above, if, indeed, mental health clinicians 
were literally only “seeing” patients (for example, waving to them as they walked past 
their cell, or intentionally conducting a perfunctory wisp of a visit when a longer visit 
was clinically necessary, for the sole purpose of checking off a box to satisfy the PM), 
one would have to consider that such a visit was disingenuous and not sufficient to satisfy 
the requirement for the patient to be “seen.” Based on my review of scores of records, 
conversations with many ADC mental health care providers, and input from Dr. 
Abplanalp, I do believe that some of the short visits are too short to be clinically 
effective, and in the context of the cases, place patients at significant risk of substantial 
harm. However, while reflecting less than adequate clinical judgment 22 ,1 do not believe 
even these clinically insufficient visits were perfunctory and disingenuous and therefore 
do not believe that they violate the spirit of “seeing” patients. 

VI. (Statistical) 

The cases I reviewed in depth were sampled from among those identified by Plaintiffs. 

To address the Court’s question of whether the reported PM perfonnance levels are 
accurate, however, requires an empiric, more statistically-oriented, analysis: Even if the 
Court were to order the Parties to apply a subjective or appropriateness test to the PMs, 
(rather than applying the objective it-happened-or-it-didn’t-happen test I believe was 
intended), for example by setting a minimum required visit length, would the PMs still be 
found in compliance? 

Because of the inherent difference I explained earlier between PMs 73, 74, 76, 78, and 
80-90 (non-watch-related PMs) and PMs 91, 94, and 95 (watch-related PMs), I analyzed 
these two groups separately to answer this question. For both analyses, I sampled from 
among the already randomly selected cases used for the CGARs, limiting the sample to 
cases that were found Compliant. 

I sampled a total of 52 cases from across the 15 non-watch-related PMs and across 
Eyman, Florence, Perryville, Lewis, Tucson, and Yuma complexes, sampling more 
heavily from the first three complexes because these were complexes highlighted by 
Plaintiffs in advocacy letters to Defendants. My test revealed the following results. For 


before by Plaintiffs and should have been included in the original protocol if that was their 
intent.” 

22 Defendants note that an alternative explanation for my observation is that the care was 
adequate but insufficiently documented. While this is possible, it is axiomatic in health care that 
if a clinical activity wasn’t documented, it wasn’t done, absent other evidence indicating that it 
was. 
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almost all 52 cases (46), the mental health professional did not record the length of the 
visit. It should be noted that mental health professionals are not required to record visit 
lengths, either by policy or according to the PMs. In the six cases where the professional 
did record the length, the visit lengths were: 40, 10, 5, 5, 5, and 2 minutes. If I were to 
arbitrarily use 10 minutes or more 23 as an acceptable visit length, i.e. assuming that 
shorter visits were Noncompliant regardless of the clinical content of the visit, two of the 
six visits for which a time was recorded would be Compliant. However, given that no 
time was recorded for 46 cases, it is impossible to know what the compliance rate would 
be using the arbitrary 10 minute cutoff: The compliance rate would be vastly different if 
all visits were included (48/52=92%) or only the visits for which a length were recorded 
(2/6=33%). 

I sampled a total of 20 cases from the three watch-related PMs at Eyman and Perryville 
complexes. I selected these two complexes because these were complexes highlighted by 
Plaintiffs in advocacy letters to Defendants 24 . My test revealed the following results. For 
about half of the 20 cases (11), the mental health professional did not record the length of 
the visit. In the cases where the lengths were recorded, they were: 10, 5, 5, 5, 5, 5, 2, 2, 
and 2 minutes. If I were to arbitrarily use 10 minutes or more as an acceptable visit 
length 25 , and count the cases where no time was recorded as Compliant, the overall 
compliance rate would be 60%. This analysis suggests that if the Court were to order that 
“seen” requires a subjective determination of the clinical adequacy of the length of each 
visit, the PM performance levels reported as Compliant in the CGARs would likely 
convert from Compliant to Noncompliant. 


23 While I think this is an arguably rational cutoff, even choosing a higher cutoff would not 
materially change the result for this set of data. 

24 Florence was a third complex highlighted by Plaintiffs. However, it has a recent history of 
noncompliance, so focusing on the other two facilities with histories of compliance would be 
expected to give a more meaningful answer to whether compliant results are accurate. Plaintiffs 
had also expressed concerns about the same issue at Fewis, Yuma, and Phoenix. However, I did 
not include them in the sample. I came across these Advocacy Fetters in my files late in the 
drafting process and chose not to include them because this would have delayed circulation of 
the draft and also, I could not envision finding any data that would alter my conclusions. Indeed, 
my review of this issue at other facilities, as noted above, already demonstrated that care 
delivered during many of these short visits was not safe and thus finding more (or fewer) 
examples of this would not be likely to change my conclusion. 

25 In their response to a draft of this report, Defendants suggest that I should use a shorter cutoff 
than 10 minutes for watch-related encounters, citing my own opinion earlier in the discussion 
wherein I stated that there are times when it is appropriate for watch-related visits to be shorter. 
There is some merit to this suggestion. Indeed, I would expect encounters in the non-acute 
setting when chronic care is being provided to generally be longer (in the range of 30 to 60 
minutes) than those in the acute watch-related setting when very narrowly focused care is being 
provided. However, it would be more appropriate to accomplish this by increasing the cutoff for 
non-watch encounters, rather than decreasing the cutoff for watch-related encounters. I have 
chosen to leave the cutoff as is because it results in a more conservative opinion. 
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VII. (Statistical) 

Based on the data I reviewed for my tests, most mental health professionals did not 
record the length of their visit, making it impossible to determine if these visits were 
Compliant or Noncompliant. Remeasuring these PMs retrospectively, restricting the 
sample to cases in which the length of the visit were documented, would be statistically 
invalid 26 . 

Recommendation 16: 

I think the weight of evidence I reviewed favors an objective (vs. subjective) approach to 
the interpretation of “seen” in mental health PMs 73, 74, 76, 78, 80-91, 94, and 95, i.e., 
that documentation of a visit with a signed progress note is sufficient evidence that a 
patient was “seen,” regardless of the length of the visit. However, because the quality of 
care delivered during short visits did, in some cases, pose a significant risk of serious 
harm , and because there are legal aspects to this issue, I do not offer a firm opinion or 
recommendation, but rather offer the Court a contingent path. 

If the Court orders the Parties to apply an objective test to “seen,” then I do not 
recommend any further action. 

If the Court orders the Parties to apply a subjective test to “seen,” I recommend these 
PMs be re-audited retrospectively in a two-step protocol. In the first step, monitors would 
apply a 10-minute cutoff: any visit with no visit time recorded 27 , or a visit time of 10 
minutes or greater is compliant with regard to “seen.” Any visit with a recorded time of 
less than 10 minutes would then be subjected to a second step: a clinical review of the 


26 It would be invalid because selecting only cases with documented times violates the rules of 
random selection. Indeed, there is reason to believe that visits in which the care giver documents 
the time (or care givers who document times) are systematically different (i.e. biased) from other 
visits. 

27 Plaintiffs presented an argument, which deserved consideration: that visits with no time 
recorded should not be included in the sample, i.e. the test would be: of the visits for which a 
time a recorded, the percentage where the visit time was 10 minutes or longer. They argue that it 
is inappropriate to assume that if no time is recorded, the visit was of acceptable length and that 
doing so “creates a perverse incentive for staff never to record the visit length, since their 
contacts will then always be counted as compliant, no matter how brief and perfunctory they 
are.” Their argument is not without some merit and underscores the complexity of this issue and 
the lack of a good solution for the retrospective review. Nonetheless, I offer the protocol I do in 
the event the Court chooses the “subjective” option, because: (a) The perverse incentive would 
not be an issue since this protocol is for a retrospective re-audit. (Modifying this PM for 
prospective measurement would likely result in a very differently worded PM.) (b) It is only that 
certain visits had short times recorded which prompted concern; to ignore the other visits which 
did not prompt concern ignores data which was not viewed as problematic, (c) Finally, as stated 
in paragraph VI and its accompanying footnote, the subset of visits in which times are recorded 
(approximately 35% of the total, based on the records of 27 Perryville patients highlighted by 
Plaintiffs) is likely a biased sample. 
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visit in the context of the patient’s overall management to determine if the visit length 
were appropriate. This second review would be conducted by a mental health clinician. 
For the reasons stated above in paragraph IV, there is a statistical drawback to a 
retrospective review. However, I believe the alternative - a prospective review, i.e., 
restarting the 24-month clock - would be onerous. 

The Defendants’ position supports the use of the objective test, which maintains the 
status quo and “was intended and highly litigated by the parties. To impose a time 
requirement now, after four years, is patently unfair and, according to Dr. Stem, “would 
likely cause an negative unintended consequence. ... At best this would unnecessarily 
waste resources; at worst it could agitate a patient who wanted to be left alone.” Further, 
there is no minimum time requirement in the community, the National Commission on 
Correctional Health Care, or the American Corrections Association.” The Plaintiffs’ 
position supports a modified version of the subjective test described in my second option. 
Their modification, discussed in the last footnote, would exclude from the scoring any 
visit with no visit time recorded. They argue that “an encounter of 5, 3, or 2 minutes does 
not satisfy either the Stipulation or the Eighth Amendment.” 28 

Seeing Patients Every “X” Days 

Issue: 

PMs 54, 61, 66, 73, 77, 80-84, 86-89, 90, and 92 share a common construct: that certain 
patients need a visit with a health care professional at some regular interval, for example 
PM 73: “All MH-3 minor prisoners shall be seen by a licensed mental health clinician a 
minimum of every 30 days.” The construct, which has been labelled the “every ‘X’ day 
issue,” has been discussed at length in Court proceedings. For the mental health-related 
PMs in this group (PMs 73, 77, 80-84, 86-89, 90, and 92), the Court, acceding to the 
Plaintiffs’ request, ordered the Parties to use the current protocol for detennining if a 
sampled case was compliant with the applicable interval. (Doc. 2225) The protocol is 
quite complex and in its complexity creates an incentive for the vendor, once they 
discover an overdue visit, to conduct extra (useless) visits to prevent repeated penalty for 
an error they have since corrected. There is some anecdotal evidence that, in fact, 

Corizon engaged in such manipulation (e.g. testimony from a Corizon mental health 
clinician (Transcript of Proceedings Evidentiary Hearing 5/31/18 at 56)). The 
manipulation apparently consists of scheduling a patient for two back-to-back 
appointments (e.g. one day and the next day). 


28 1 include for completeness, Plaintiffs’ support for this assertion (Disability Rights Montana v. 
Batista, 930 F.3d 1090, 1094 (9th Cir. 2019), complaint alleging, inter alia, that prisoners’ 
“primary contact with mental health staff... lasts no more than a few minutes” stated an Eighth 
Amendment claim)” and Defendants’ assertion that the supporting case “does not stand for the 
proposition that there is a minimum amount of time needed for a MH contact in order to satisfy 
the Eighth Amendment.” 
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Plaintiffs have raised a concern that the reported performance levels of the 16 above-cited 
PMs may therefore not be accurate. Thus I addressed this issue in my review. I limited 
my review to most of the mental health-related PMs in the group (i.e., PMs 73, 77, 80-84, 
87-89, 90, and 92). I excluded PMs 54, 61, 66, and 86, for the following reasons: (1) 
There is no anecdotal evidence that vendor manipulation has occurred outside of the 
mental health PMs. (2) The non-mental health PMs do not use the same complex protocol 
and are thus less susceptible to manipulation. (3) I reviewed PM 54, which deals with 
chronic care visits, elsewhere. (4) For PM 61, which deals with Pap smears, I considered 
it unlikely that clinicians would perform, or that patients would consent to, back-to-back 
unnecessary pelvic examinations. (5) Although it is a mental health-related PM, I deal 
with PM 86 elsewhere. 

With Dr. Abplanalp’s assistance, I sampled a total of 107 cases across 12 PMs and across 
Douglas, Eyman, Florence, Perryville, Phoenix, Lewis, Tucson, and Yuma complexes. 
The details of the methodology are described in the footnote. 29 My test revealed the 
following results. Among the first 57 cases, I found a single case (Patient 29) where an 
extra visit was conducted on 11/08/18 with the apparent sole purpose of avoiding 
Noncompliance with PM 77 ( Mental health treatment plans shall be updated a minimum 
of every 90 days for MH-3A, MH-4, and MH-5 prisoners, and a minimum of every 12 
months for all other MH-3 prisoners.). While, as I understand it, the case would have 
been found Noncompliant if measured during a certain month’s audit, with or without the 
extra visit, the extra visit prevented the vendor from being found Noncompliant during 
subsequent months, even though the error had been corrected. As a result of this single 
finding among the first 57 cases, I additionally reviewed all 50 of the remaining 


29 The vast majority of cases were drawn from the December, 2018 CGAR. However, because of 
the absence of any cases at Phoenix for many of the PMs during December, 2018,1 used the 
January, 2019 CGAR for all but three of the months with no cases. For these three months there 
were no cases at Phoenix in December, 2018 or January, 2019, so I drew the samples from the 
most recent months for which there were cases (May, 2018 for PM 82, November, 2017 for PM 
83, and June, 2017 for PM 84). I generally chose the first one or two Compliant cases in each 
yard published in the CGAR I was working with. As a first step, I applied this methodology to 
PMs 73, 77, 80, 83, 84, 87, 88, 89, 90, and 92. For efficiency, I deferred review of PM 81 ( MH3- 
A prisoners who are prescribed psychotropic medications shall be seen a minimum of every 90 
days by a mental health provider.) and PM 82 {MH-3B prisoners shall be seen a minimum of 
every 90 days by a mental health clinician.) until the latter part of the review because I judged 
that the constructs of review of MH-3A patients, MH-3B patients, provider visits, and clinician 
visits, were going to be well covered in the tests of the rest of the group of PMs, and if the error I 
was testing for did not significantly appear in the tests of the rest of the group, it was unlikely to 
appear in these. As this turned out to be the case (only one concerning finding among 57 cases), I 
limited further testing of PM 81 and 82 to two cases each at Phoenix in January 2019, which had 
already been completed at that point. 
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Compliant cases from that complex (Phoenix) during the same month. I found no further 
similar cases. 

My analysis leads me to conclude that: (a) manipulation does happen 30 ; (b) it is driven by 
an unintended consequence of the complex protocol for measuring these PMs (and is a 
key reason for my recommendations elsewhere for revision of these PMs); and (c) the 
PM performance levels, as reported, are accurate. 

Recommendation 17: 

Except as cited elsewhere in my report, the reported performance levels of PMs 54, 61, 
66, 73, 77, 80-84, 87-89, 90, and 92 should be accepted as accurate with regard to the 
“every ‘X’ day” issue. 

The Parties concur except that Plaintiffs “do not agree that the reported scores are 
accurate to the extent that they count as “compliant” encounters lasting 5 minutes or 
less,” consistent with Plaintiffs’ comments regarding Recommendation 16. 


Issues Specific to a Single PM 

PM 25 (A first responder trained in Basic Life Support responds and adequately provides care 
within three minutes of an emergency.) 

Issue 1: 

This PM was written in response to an event a few years ago in which a mentally ill 
patient cut himself with a razor, COs failed to summon health care staff in a timely 
manner or provide basic first aid to stop the bleeding, and the patient died. 

While the intent of the PM was to assess the initial response by the first person 
responding to the scene who is trained in first aid, although that person is typically a CO, 
on rare occasion it is a nurse. The worst performing facility has been Lewis. A review of 
the Noncompliant cases there revealed that the monitor (someone no longer working at 
ADC) found performance Noncompliant due to failure of nurses to follow emergency 
response protocols (the specific deficiencies are not listed). Nurses are generally not the 
first responders in ADC nor the intended target of the PM. No deficits in the CO response 
were noted. Thus the “Noncompliant” performance levels assigned to Lewis for 
December 2017, April 2018, and June 2018 are erroneous. 

Recommendation 18: 

The performance levels for Lewis Complex on PM 25 for December 2017, April 2018, 
and June 2018, should be changed to “Compliant.” 


30 In their comments to the draft of this report, the Defendants assert that extra visits are very 
rare, and therefore “do not agree with the conclusion that manipulation is occurring.” 
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The Defendants concur. The Plaintiffs do not concur. They believe this PM should 
measure both the care provided by COs and subsequent care provided by medical staff 
(nurses). As such, what was measured at Lewis (and found Noncompliant) was exactly 
what should have been measured. 

Issue 2: 

Plaintiffs allege that monitors disregard the “adequacy” component of this PM. As noted 
above, the “adequacy” component is limited to the first responder who is almost always 
a CO. Based on my interviews with a number of the monitors who audit this PM, the 
methodology used by the monitors varies by monitor. Some monitors (as described 
above) go beyond the four comers of the PM. Some attempt to assess the adequacy of CO 
responses by reviewing CO documentation of the emergency response as described in the 
COs’ Incident Reports and by reviewing any video recordings of the emergency 
response. But at least one limits her review to the time component of the PM. 

Beyond the problem of heterogeneity in the auditing of this PM, there are significant 
challenges in measuring it. First, not all emergency responses are video recorded. This is 
because they may occur in a location that is not video surveilled (e.g. the interior of a 
cell). Second, based on my experience, CO reports are generally unreliable windows 
through which to examine the adequacy of medical care. Third, it is very difficult to 
assess the adequacy of medical care delivery via video, especially when there is no sound 
or poor sound quality, which is usually the case. Finally, given the infrequency of 
emergency responses in ADC complexes 31 , the results of this PM are not statistically 
meaningful. For example, in the month of December, 2018, there were a total of 11 
events statewide. These occurred at six facilities with the following frequency per 
facility: 3; 3; 2; 1; 1; 1. Performance levels based on such small numbers are highly 
unreliable. 


11 In their comments on a draft of this report, Plaintiffs note that “there are dozens of ICSes at 
each prison each month. Defendants have defined emergency response to mean only when a 
person needs CPR or [is] bleeding.” Plaintiffs are correct. However, the only other emergency to 
which PM 25 appears to apply, based on the Methodology section of PM 25 in the Monitoring 
Guide, is “stabilization of injuries and wounds.” Based on my experience, it is even more 
difficult to assess the quality of medical care delivered for injuries and wounds, rendering 
performance results unreliable. Further, it is unlikely that the addition of emergency responses 
for “injuries and wounds” would solve the problem an unreliable measure due to small numbers. 
Nonetheless, I have incorporated the need to include emergency responses for injuries and 
wounds in the future in the event the Court does not adopt Option 1 of Recommendation 18 
calling for retirement of PM 25. 
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Recommendation 19: 

Option 1 

For the reasons stated above, I believe PM 25 is an unreliable measure and recommend it 
be retired. (Even if this Recommendation is not approved, I still recommend termination 
of this PM in the facilities that have met the requirements for termination - see Part IB.) 

Option 2 

If Option 1 is not acceptable to the Court, PM 25 should be recalculated as follows. ADC 
should determine the method used by each of the monitors who audited PM 25 over the 
prior 24 months. If the monitor did not limit his or her review to emergency response of 
the first responders, i.e., COs, those months should be re-audited to exclude cases where 
the care delivered by the second responder (i.e. nurses) was evaluated. In addition, going 
forward, monitors should include review of emergency responses due to injuries and 
wounds. 

The Defendants concur with Option 1. They do not concur with Option 2 “as the reports 
of incidents may not include an adequate description of medical care provided by the first 
responders (typically COs) in order to evaluate whether “appropriate” medical care was 
rendered. COs are trained in first aid only. The IRs generally do not include detailed 
discussions of emergency medical care provided.” Defendants’ comments echo my 
concerns about this PM. The Plaintiffs do not concur with either option, holding instead 
that PM 25 should “be modified to actually evaluate the adequacy and timeliness of 
emergency responses by first responders whether they be officers or medical staff.” In 
addition, they reiterate their recommendation for a more robust definition of adequate 
first aid, proffered in Doc. 1561 at 11-14, borrowed from the California prison system. 
While the California definition is clinically sound and a desirable goal, in my opinion it is 
subject to the same challenges I discuss above. 

PM 44 ( Inmates returning from an inpatient hospital stay or ER transport with discharge 
recommendations from the hospital shall have the hospital’s treatment recommendations 
reviewed and acted upon by a medical provider within 24 hours.) 

Issue: 

Measurement of this PM is inaccurate for a few reasons, and as a result, the reported PM 
levels may be materially over- or understated. An overlying problem is that the PM is 
audited by many different monitors and there is large variation in how they interpret and 
audit it. Another overlying problem is that auditing this PM requires clinical judgment, 
often at the level of a provider; however, the PM is audited by nurses. I found instances 
in which the monitor found a case Noncompliant when it should have been found 
Compliant, e.g. (a) a facility provider acted upon a hospital recommendation in a manner 
that was effectively consistent with the recommendation, but not exactly what had been 
recommended, or (b) a facility provider failed to act upon a hospital recommendation 
which should not have been acted upon. Conversely, I found instances in which the 
monitor found a case Compliant when it should have been found Noncompliant, e.g. a 
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facility provider acted upon a recommendation which was inappropriate and should have 
been ignored. Some monitors did not examine whether the facility provider’s order was 
actually acted upon (carried out) 32 . Some monitors failed to examine the specific 
discharge recommendations and simply considered the case Compliant if the facility 
provider was contacted or saw the patient within 24 hours of return from the hospital. 
Finally, in some instances, cases were found Compliant when valid recommendations 
were simply not acted upon. In summary, I found scoring errors of both over- and 
understatement, and scoring errors both remediable by more nurse-monitor consistency 
and not remediable without clinical judgment from a provider-monitor. I therefore 
believe the existing results for PM 44 may not be accurate and should not be relied upon. 

In recognition of the fact that Defendants have expressed an objection to modifications to 
PMs whereby the PM would require clinical judgment, I have created two options. 

Option 1 calls for re-audit to correct any errors due to inconsistency in monitoring (within 
the scope of a nurse-monitor) by nurse monitors; no providers or clinical judgment would 
be used. Option 2 calls for re-audit to correct errors covered by Option 1 plus errors due 
to the use of nurses without input from a provider. Having two options allows the Court a 
cleaner way to accept or reject Defendants’ objection to the use of clinical judgment. 
Option 2 is more complete, more clinically appropriate, and therefore, in my opinion, the 
better option. In recognition of the fact that some errors may have resulted in 
understatement of performance levels, I am also offering a recommendation to allow 
Defendants to re-audit Noncompliant months (i.e. possibly turning Noncompliant months 
into Compliant ones). 

Recommendation 20: 

Option 1; 

I recommend that PM 44 be retrospectively re-audited for complexes/months 
reported as Compliant upon which ADC intends to rely for termination of the PM. When 
examining each case, the (nurse) monitor should address: 

1. Did the patient return from the hospital with at least one discharge 
recommendation? If not, the case should be skipped and replaced by the next 
randomized case. 

2. Was each discharge recommendation ordered by the facility provider within 24 
hours? 

3. If not, is there “a documented reason explaining why the prescribed treatment 
was rejected.” . 

4. Did the patient “receive[ ] the prescribed treatment”? (Doc. 1831 at 2) For the 
purposes of this step, the Court’s order should be interpreted as follows: First, 


32 This requirement was added by the Court on 12/14/16: “... [T]he Monitoring Guide shall only 
permit compliance if the inmate received the prescribed treatment, or if there is a documented 
reason explaining why the prescribed treatment was rejected.” (Doc. 1831 at 2) 
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“prescribed treatment” should include treatments as well as tests, consultations, or 
any other executable provider order. Second, the monitor should consider the 
response to the recommendation Compliant if there was a provider order, and the 
order was scheduled, but was not consummated because it was scheduled to take 
place at a date after the date of the audit. 

Option 2: 

I recommend that PM 44 be retrospectively re-audited for complexes/months reported as 
Compliant upon which ADC intends to rely for termination of the PM. The re-audit 
should be conducted by, or in collaboration with, a provider. When examining each case, 
the monitor should address: 

1. Did the patient return from the hospital with at least one discharge 
recommendation? If not, the case should be skipped and replaced by the next 
randomized case. 

2. Was (were) the discharge recommendation(s) clinically appropriate? 

3. If so, was each ordered by the facility provider within 24 hours? 

4. If not, is there “a documented reason explaining why the prescribed treatment 
was rejected.” (Doc. 1831) The Court’s order should be interpreted in a practical 
manner: “Documentation” should not be limited to a specific statement by the 
facility provider explaining why a recommended treatment was rejected, but 
rather should include documentation in the patient’s medical record which the 
facility provider was likely aware of and which render the reason for rejection 
obvious. For example, if the hospital physician recommends starting the patient 
on penicillin, the patient’s ADC medical record clearly shows a serious penicillin 
allergy, and the facility physician orders the patient started on a different, but 
equally effective, antibiotic, the monitor should show that case Compliant with 
PM 44 as modified by the Court. 

5. For orders resulting from recommendations (or appropriate modifications of 
recommendations, as described in (4) above), did the patient “receive[ ] the 
prescribed treatment”? (Doc. 1831) For the purposes of this step, the Court’s 
order should be interpreted as follows: First, “prescribed treatment” should 
include treatments as well as tests, consultations, or any other executable provider 
order. Second, the monitor should consider the response to the recommendation 
Compliant if there was an appropriate provider order, and the order was 
scheduled, but was not consummated because it was scheduled to take place at a 
date after the date of the audit. 

The Plaintiffs agree with Option 1 and Option 2. The Defendants disagree with 
Option 2, explaining that my “recommendation for a new protocol is outside the 
parties’ Stipulation. The parties contracted for an objective approach to measure a 
period of time, and Dr. Stem’s recommendation to engage a provider for 
subjective/difference in medical opinion is contrary to their intent. To implement 
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Dr. Stern’s recommendations, the Stipulation would need to be revised. 
Defendants do not agree to such a revision.” There is merit to the Defendant’s 
explanation because, as stated elsewhere, the Stipulation is almost devoid of PMs 
that require clinical judgment; the changes I recommend here would not follow 
that pattern. The Defendants also disagree with Option 1, citing the same 
reasoning. For the reasons explained in the discussion of the issue, Option 1 is an 
objective approach and does not require professional subjective opinion, so the 
Defendants’ disagreement with it appears inconsistent with their goals and 
position. The Defendants’ second reason for disagreeing with Option 1 is that 
Item 4 “requires a determination of whether the treatment was actually performed. 
Item 4 is not in the protocol. The protocol only requires a determination of 
whether the recommendation was acted upon, not whether the recommendation 
was completed.” The Defendants are correct. The “protocol” (Monitoring Guide) 
does not include Item 4. However, Item 4 was added by the Court. Finally with 
regard to any re-audit, Defendants “do not agree that past results are unreliable 
and should be re-audited” but did not offer any specific methodological flaw in 
my analysis. 

Recommendation 21: 

ADC should be allowed, at its discretion, to also retrospectively re-audit PM 44 
according to the methodology in the previous Recommendation, for any complex/months 
previously reported Noncompliant. 

The Plaintiffs concur. The Defendants do not concur, citing that “Dr. Stem’s 
recommendation for a new protocol is outside the parties’ Stipulation. The parties 
contracted for an objective approach to measure a period of time, and Dr. Stem’s 
recommendation to engage a provider for subjective/difference in medical opinion is 
contrary to their intent. To implement Dr. Stern’s recommendations, the Stipulation 
would need to be revised. Defendants do not agree to such a revision.” Defendants also 
“do not agree that past results are unreliable and should be re-audited” but did not offer 
any specific methodological flaw in my analysis. 

PM 46 (A Medical Provider will review the diagnostic report, including pathology reports, and 
act upon reports with abnormal values within five calendar days of receiving the report at the 
prison.) 

Issue: 

Measurement of this PM is inaccurate because it utilizes the wrong unit of analysis (see 
UoA discussion above). The random sample for this PM includes a large number of 
diagnostic reports which reside within the same patient. For example, of the 10 diagnostic 
reports sampled for the Central yard at Florence for December, 2018, four tests came 
from one patient, two tests came from another patient, and the remaining four came from 
four other patients. The various diagnostic reports tests on the same patient - especially 
when they are reported back on the same day at the same time (see, for example, 
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“Diagnostic Panel 2” and “Hepatitis A” tests for Patient 53, reported 12/06/18, appearing 
on December, 2018 CGAR for Central yard) - are closely “related” to each other. In 
statistical terms, they are not independent of each other. Whatever the provider did with 
regard to one report (i.e., acted upon it timely or failed to act upon it timely), there is a 
better-than-even chance that he or she did the same thing with regard to a second or third 
report. This scientific error could bias the accuracy of the PM performance in either 
direction: a facility which failed the PM may be more compliant than reported, and a 
facility that passed the PM may be less compliant than reported. Due to the frequency 
with which multiple reports were sampled within the same patient, I believe the existing 
results for PM 46 may not be accurate and should not be relied upon. 

Recommendation 22: 

I recommend that PM 46 be retrospectively re-audited for complexes/months reported as 
Compliant upon which ADC intends to rely for termination of the PM. The following 
protocol should be followed. The first diagnostic report for each patient within a month’s 
CGAR sample should be retained. Any additional reports for the same patient should be 
discarded and replaced with the next report for the next unique patient on the original 
randomized Source Document for that CGAR, until the quota of 10 samples is reached. 

In the unlikely event that there are an insufficient number of unique patients with to fill 
the quota of 10 samples, the auditor will draw the second diagnostic report (if there is 
one) from each patient already sampled (and then, if necessary, the third diagnostic 
report, and so on) until the sample size is 10. 

The Plaintiffs concur. The Defendants do not concur. The Defendants “do not agree that 
past results are unreliable and should be re-audited” but did not offer any specific 
methodological flaw in my analysis. Further, the Defendants disagree with a change to 
the unit of analysis from the diagnostic report to the patient. The measure is set up to use 
the diagnostic report as the unit of analysis. If there is a change to using the patient, it 
will reduce the sample size, which will make the results statistically unreliable.” In 
response to this latter concern, I modified the recommendation to deal with the 
eventuality of a small sample size, however, the Defendants wish this objection to stand. 

Recommendation 23: 

Going forward, the same protocol should followed as in the recommendation above, 
using the current month’s Source Document. 

The Plaintiffs concur. Defendants “do not agree that past results are unreliable and should 
be re-audited” but did not offer any specific methodological flaw in my analysis. The 
Defendants also do not concur due to the change in unit of analysis. (The Defendants’ 
reasoning and my response are the same as in Recommendation 22.) 
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Recommendation 24: 

ADC should be allowed, at its discretion, to also retrospectively re-audit PM 46 
according to the methodology in the previous Recommendation, for any complex/months 
previously reported Noncompliant. 

The Plaintiffs concur. Defendants “do not agree that past results are unreliable and should 
be re-audited” but did not offer any specific methodological flaw in my analysis. The 
Defendants also do not concur due to the change in unit of analysis. (The Defendants’ 
reasoning and my response are the same as in Recommendation 22.) 

PM 54 ( Chronic disease inmates will be seen by the provider as specified in the inmate's 

treatment plan, no less than every 180 days unless the provider documents a reason why a 

longer timeframe can be in place.) 

and 

PM 55 ( Disease management guidelines will be implemented for chronic diseases.) 

Issue: 

First, PM 54 is largely duplicative of PM 55. PM 55 is more comprehensive and requires 
more disease-tailored time intervals than the blanket 180 days requirement for all 
diseases in PM 54. Thus there is no added value in measuring PM 54. Further, the same 
case can be found Compliant under one PM but not under the other. For example, a 
patient with latent tuberculosis who is seen at an interval of one year, can be found 
Noncompliant under PM 54 (which requires visits every 180 days) and Compliant under 
PM 55 (which requires visits every year). Second, the Source Document is comprised of 
patients who carry a chronic disease diagnosis, regardless of whether they have had a 
chronic care visit or when. This is good, because it would capture patients for whom no 
chronic care visits have taken place (which would be a serious error). However, more 
commonly, it includes patients who have not yet reached the time interval for their next 
visit, and therefore could not possibly be found Noncompliant. Third, PM 55 is audited 
differently by different monitors. All monitors audit to the stated requirements of the PM, 
but some audit beyond the requirements, for example, checking whether the provider has 
also ordered blood tests. Fourth, also as a result of the third issue above, some monitors’ 
expectations for performance of activities, such as blood tests, exceed clinical 
requirements. For example, a monitor found the case of a patient with hepatitis C (Patient 
35) at Lewis Noncompliant because the patient did not have repeat blood tests at his most 
recent chronic care clinic visit, even though no such testing was clinically necessary. 

Fifth, the Source Document is comprised of a separate entry for each chronic disease for 
each patient, i.e., the UoA (Unit of Analysis) is the condition, not the patient. As a result, 
the same patient may be randomly selected more than once. The management of different 
chronic diseases within a given patient are not likely to be independent of each other. For 
example, two or more chronic diseases are - wisely - often managed at the same visit. 

For this reason, the UoA for these PMs should be the patient, not the disease. While some 
of the effects of these issues are unpredictable, in my opinion, the net effects are that (1) 
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the reported level of performance on PM 55 may be understated, and (2) the audit process 
is wasteful of resources. 

I discuss the aspects of chronic care that are not currently monitored by PM 55 (or 54) in 
Part IV. 

Recommendation 25: 

Going forward, PM 54 should be retired in favor of PM 55. PM 55 should be audited 
according to the following protocol. The overall concept of the protocol is that it checks 
both to see if providers are planning for follow-up as well as whether staff are executing 
those plans. 

1. ADC should continue to use as a Source Document a randomized list of all 
chronic condition-patient pairs extant on the last day of the audit month. 

2. If a patient appears on the list more than once, the first instance should be 
audited; subsequent appearances should be skipped. In the unlikely event that 
there are an insufficient number of unique patients with to fill the quota of 10 
samples, the auditor will draw the second chronic disease (if there is one) from 
each patient already sampled (and then, if necessary, the third chronic disease, 
and so on) until the sample size is 10. 

3. A case is Noncompliant if any of the following are found (and there is no 
evidence of an adequate patient refusal to explain the missing clinic visit): 

a. There are no previous chronic care clinic visits for the chronic disease 
recorded in eOMIS, and more than one month has transpired since the 
patient was admitted to ADC. (This latter specification recognizes that 
newly admitted patients will not have a previous chronic care clinic visit, 
but should have been seen for their chronic disease by the end of their first 
month of residency.) 

b. The most recent chronic care clinic visit did not occur within the time 
frame ordered by the provider at the previous visit; if no time frame was 
ordered, it occurred more than six months after the previous visit. 

c. A follow-up chronic care clinic visit is not scheduled in the future (i.e., 
beyond the end of the audit month) 

d. A follow-up chronic care visit is scheduled in the future (i.e. beyond the 
end of the audit month) but the date of that future appointment is not 
consistent with the provider’s order at the most recent chronic care clinic 
visit; if no time frame was ordered, it is scheduled more than six months 
after the most recent visit. 

The Plaintiffs concur. The Defendants concur with retirement of PM 54. The Defendants 
do not concur with any changes to the method of measuring PM 55, explaining, 
“Noncompliance cannot be determined based upon a future visit, because the visit has not 
yet taken place. Just because a visit has not been scheduled, does not mean that it will 
not be scheduled. Delete “in favor of PM 55”. The Defendants raise a valid point. 
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Theoretically, the future visit could occur, even though it was not ordered and scheduled 
at the time of the previous visit. However, part of the purpose of a chronic care visit is to 
plan future care based on the patient’s current status. Further, based on my experience, if 
the next visit is not thought about and planned at the previous visit, it almost never takes 
place in a reasonable amount of time. Finally, though the method I recommend does 
introduce the imperfection cited by the Defendants, it is small in relationship to the larger 
problems with the current method that it resolves. 

Recommendation 26: 

ADC should be allowed, at its discretion, to also retrospectively re-audit PM 55 
according to the methodology in the previous Recommendation, for any other previously 
reported Noncompliant complexes/months. 

The Parties concur. The Defendants add that “Should the re-audit establish compliance... 
the measure should be terminated.” 

PM 85 ( MH-3D prisoners shall be seen by a mental health provider within 30 days of 
discontinuing medications.) 

Issue: 

PM 85, and its closely related measure PM 86 (see below) have been the subject of 
tremendous discussion between the Parties and have consumed much of the Court’s time 
to the frustration of all. Despite the well-intentioned efforts of all, the PMs remain 
problematic. The Parties still have disparate ideas of what the Court ordered as well as 
ideas of what is actually being done. Without revisiting the lengthy history of these PMs, 
in brief, both PMs are currently being measured in a scientifically flawed manner. The 
major flaw is that the same case may be (and sometimes is) selected on more than one 
month’s audit to measure the same event. When selected, however, the scoring of that 
event is accurate. Thus the resultant scores are not necessarily wrong, but they are 
statistically “watered down.” I therefore must conclude that the reported performance 
levels for these PMs are not reliable. 

Recommendation 27: 

ADC should retrospectively re-audit PM 85 according to the protocol below, for any 
month ADC intends to use as evidence of compliance and which is currently Compliant. 
As a first pass, the re-audit should be limited to every third month. (If that third month 
was previously found Noncompliant, the next closest Compliant month should be 
chosen. 33 However, ADC may, at its discretion, re-audit the skipped - or any other - 

33 In their response to a draft of this report, the Defendants posit “that a month should not be 
excluded because the third month is noncompliant. If the re-audit turns the month to compliant, 
it should be used as a basis for termination of the measure.” The driver for recommending re¬ 
audit is that the previous audit method might have resulted in over-statement, not under¬ 
statement of compliance results, thus it is appropriate to only choose Compliant months in the 
three-month approach. However, the Defendants should not be barred from looking for and 
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Noncompliant month.) If, however, this re-audit results in any single month moving from 
Compliant to Noncompliant, all relevant months should be re-audited for that complex. 34 

For clarity and simplicity, I use the month of July as the audit (CGAR) month. 

1. The Source Document is the list of all patients who were listed as being MH- 
3D (“have been recently taken off of psychotropic medications and require follow 
up to ensure stability over time”) during the month of July. 

2. Randomize the list. 

(Randomization may be done at any point before selection of cases; I have 
arbitrarily placed it as step 2.) 

3. Exclude all patients except those whose medications were stopped in the month 

of June. 

(Excluding cases may be done at any point; I have arbitrarily placed it as 
step 3.) 

(If the provider instructed the patient to taper the medication rather than 
stop it abruptly, the date of stoppage is the last date the patient took the 
medication. If the patient stopped the medication some time earlier on 
his/her own, the date of stoppage is the date of the visit with the provider 
when the provider acknowledges and agrees with the stoppage.) 

4. Select the first 10 patients in each yard. 

(If the patient moved from one yard or facility to another during the 
month, the case should be audited under the yard where the patient resides 
at the end of July, the day the Source Document is produced.) 35 


benefiting from under-statement. For this reason, in response to the Defendants comment, I have 
modified the protocol to allow ADC, at its discretion, to re-audit Noncompliant months. 
l4 I propose a “third-month” approach here and elsewhere. The overall goal of this method is to 
reduce the workload of re-auditing while preserving the validity of the re-audit. I reserve this 
approach for PMs where there is evidence that the reported performance levels may not be 
accurate, but that evidence is less strong than for PMs where I recommend that all (Compliant) 
months be re-audited. I chose three months because one of the Stipulation requirements for 
termination of a PM is that a PM may not have three Noncompliant months in a row. By auditing 
every third month, we will know that that specific requirement of the PM has been satisfied. 
Under the “third-month” method, the oldest (first) month to audit is the month that is 24 months 
prior to the month in which this recommendation is approved by the Court, followed by the 
fourth month, and so forth. 

35 This is an imperfect method for attributing successes or failures in follow-up to a yard. The 
more perfect method would be the following. If the patient moved from one yard or facility to 
another during the month, and the follow-up is compliant, the case would be audited under (i.e. 
“credit given ”) to the yard that conducted the compliant visit. If the follow-up is noncompliant, 
the case would be audited under the yard where the patient is living on the date the follow-up 
was due; this is the yard which had the final responsibility to ensure follow-up. When 
reassigning the case from one yard to another, the patient’s random number would dictate 
whether or not the patient will be one of the 10 chosen cases. However, after discussions with 
ADC, it is clear that given the currently available tools for generating the Source Document and 
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5. Test whether the patient had a follow-up visit with a mental health provider 
within one month 36 of stoppage. If so, the case is Compliant. 

(If any event occurs during the one-month look-back that would obviate 
the need for, or prevent the possibility of, a mental health provider visit 
(e.g. the patient dies, the patient is transferred to a private ADC prison or a 
jail, the patient is released), and therefore the follow-up did not take place, 
the case will replaced by the next randomized case in the list. If, however, 
the follow-up did take place before the event took place, the case should 
be included. 37 ) 

(Whether or not the provider restarts medication at this visit is irrelevant; 
it is the timing of the visit which determines its compliance.) 

(The fact that not all months have 30 days means that some cases may not 
get selected for audit and some cases may be selected for, arguably, the 
“wrong” month. These events should be rare. Neither “error” is material 
nor would reasonably be expected to lead to biased results, especially 
when the same methodology is applied month after month. Therefore, in 
the interest of simplicity, this one-month rule should serve well.) 

The Parties concur. 

Recommendation 28: 

If the results of this retrospective re-audit are substantially Compliant, as defined in the 

Stipulation, PM 85 should be terminated. 


conducting the audit, the more perfect method would be onerous. And given the fact that 
Noncompliant cases will be brought to light under either method, and that therefore the imperfect 
method results in a fair profile of care delivery, I am only recommending that (a) ADC explore 
ways of improving its information systems so that they might be able to use the more perfect 
method in the future, and (b) monitors record in their notes, the yard/facility responsible for 
failing to conduct the follow-up, if that yard is different from the audited yard. 

36 In accordance with the Court’s decision (Doc 1673 at 2), here, and throughout this discussion 
of PMs 85 and 86,1 use the term “month” to denote a period of time of 28, 30, or 31 days, 
depending on the month. So it is a month from January 15 to February 15, from February 15 to 
March 15, and from April 15 to May 15. 

37 Arguably, including cases that are compliant, but excluding those where there was no 
opportunity to be Compliant or Noncompliant, introduces a statistical imbalance favoring a 
finding of compliance. However, the alternative - excluding any case where the time period was 
cut short, regardless of whether a follow-up took place - could have an unintended consequence. 
The vendor, realizing that they get no “credit” for following up on a patient they know will be 
leaving soon, would be disincentivized to have the patient seen. Given these tradeoffs, I believe 
the method I prescribe is in the best interest of patient safety. 
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The Defendants concur. The Plaintiffs concur. However, the Plaintiffs repeat their 
position, explained in more depth regarding Recommendation 16, that encounters lasting 
five minutes or less should not be counted as Compliant. 

Recommendation 29: 

If PM 85 is not terminated, going forward, PM 85 should be measured in the manner 
described above in Recommendation 27. 

The Parties concur. 

PM 86 ( MH-3D prisoners shall be seen a minimum of every 90 days by a mental health clinician 
for a minimum of six months after discontinuing medication.) 

Issue: 

(See PM 85 above.) I am also recommending retrospective re-audit of PM 86. Because 
PM 86 is a little more complex to audit than PM 85,1 am recommending two protocols, 
one for auditing retrospectively, and one for auditing prospectively. My reason for 
recommending a different protocol for the retrospective period is that it is more efficient 
for the Defendants, and it more closely matches the original intent of the PM by the 
Plaintiffs. In brief, the retrospective audit looks back at the entire completed six months 
of care after a patient’s medications were stopped and examines whether all (generally 
two, one every 90 days) post-stoppage visits with a clinician took place. The prospective 
audit looks back at each of the 90 day periods separately, and doesn’t require waiting 
until a 6-month course of treatment is complete; it is thus a more real-time management 
tool. 

My recommended prospective protocol is not perfect, but it is practical, and most 
importantly achieves the intent of the PM. In its imperfection, the protocol has drawbacks 
for both Parties. For the Defendants, it will fail to give them “credit” for patients who are 
seen much earlier than the 90 day requirement. However, these events are not that 
common. For the Plaintiffs, it will sample fewer events. 38 However, Plaintiffs’ preferred 
protocol is ill-advised from a management and a patient safety perspective and therefore 
would not serve ADC well after resolution of the Stipulation: it delays auditing quality of 
care delivery by up to seven months. If, indeed, clinicians are failing to appropriately 
follow up patients, it is wrong to wait so long to find out and implement corrective action. 
With regard to the statistical issue (see previous footnote), monitors will still be auditing 
on average, around 40 to 50 individual events per complex per month, every month, and 
thus, in my opinion, the modified PM protocol I recommend is adequately powered to 


’ 8 The Plaintiffs’ preferred protocol entails examining each patient’s entire 180 day follow-up 
period after it is completed and querying whether the 10 selected patients have each had their 
two (the first and the second) 90-day visits. Thus, in effect, their method tests 20 events, not 10. 
My proposed method measures a single (the first or the second) 90-day follow-up, and therefore 
only queries 10 events. 
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detect a problem. Further, the number of events audited in the modified protocol (10 per 
yard) is consistent with the number of events audited in all other PMs which measure 
follow-up intervals. 

Finally, auditing of PM 86 is currently somehow “coupled” to the auditing of PM 85, 
according to one interpretation of the Monitoring Guide. My recommended protocol does 
not include or imply any “coupling.” 

Recommendation 30: 

Retrospective re-audit protocol: 

ADC should retrospectively re-audit PM 86 according to the protocol below, for the 24- 
month period it plans to use for evidence of Compliance. As a first pass, the re-audit 
should be limited to every third month 39 . (If that third month was previously found 
Noncompliant, the next closest Compliant month should be chosen. 40 However, ADC 
may, at its discretion, re-audit the skipped - or any other - Noncompliant month.) If, 
however, this re-audit results in any single month moving from Compliant to 
Noncompliant, all relevant months should be re-audited for that complex. If the third 
month is already Noncompliant, pick the next closest compliant month. 

For clarity and simplicity, I use the month of July as the audit (CGAR) month. 

1. The Source Document is all patients who were listed as being MH-3D (“have 
been recently taken off of psychotropic medications and require follow up to 
ensure stability over time”) during the month of January. 

2. Randomize the list. 

(Randomization may be done at any point before selection of cases; I have 
arbitrarily placed it as step 2.) 

3. Exclude all patients except those whose date of stoppage was in January. 

(Excluding cases may be done at any point; I have arbitrarily placed it as 
step 3.) 

(If the provider instructed the patient to taper the medication rather than 
stop it abruptly, the date of stoppage is the last date the patient took the 
medication. If the patient stopped the medication some time earlier on 
his/her own, the date of stoppage is the date of the visit with the provider 
when the provider acknowledges and agrees with the stoppage.) 

4. Select the first 10 patients in each yard. 

(If the patient moved from one yard or facility to another during the 
month, the case should be audited under the yard where the patient resides 


39 Under the “third-month” method, the oldest (first) month to audit is the month that is 24 
months prior to the month in which this recommendation is approved by the Court, followed by 
the fourth month, and so forth. 

40 See footnote 33. 
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at the end of January which is the day the Source Document is 
produced.) 41 

5. Test whether the patient had follow-up visits with a mental health clinician at 
least as often as every three months 42 , from the date of stoppage until six months 
later. In other words, between the date of stoppage and the date six months later, 
there must be no interval longer than three months without the patient being seen. 
If so, the case is Compliant. 

(If any event occurs during the 6-month look-back that would obviate the 
need for, or prevent the possibility of, a mental health clinician visit (e.g. 
the patient restarts medication, the patient dies, the patient is transferred to 
a private ADC prison or a jail, the patient is released), the case will be 
audited up to the point of this terminating event. 43 )(The fact that not all 
months have 30 days means that some cases may not get selected for audit 
and some cases may be selected for, arguably, the “wrong” month. These 
events should be rare. Neither “error” is material nor would reasonably be 
expected to lead to biased results, especially when the same methodology 
is applied month after month. Therefore, in the interest of simplicity, the 
use of a 7-month look back should serve well.) 

The Plaintiffs concur. Defendants “do not agree that past results are unreliable and should 
be re-audited” but did not offer any specific methodological flaw in my analysis. They 
“maintain that they have been substantially compliant with this measure and it should be 
terminated pursuant to the terms of the Stipulation.” 


41 See footnote 34. 

42 See footnote 35. 

4l The logical result of this part of the protocol is that for part (or rarely, the entirety) of the look- 
back period, the ADC may be “credited” for compliance when it could not possibly be 
Noncompliant. For example, if medications were stopped on 1/15, the patient then had his first 
clinician visit on 4/14, and then was released from custody on 5/15, the case would be found 
compliant, even though for the period from 4/15 to 5/15, ADC could not have possibly been 
found Noncompliant. Nonetheless, I make the recommendation to audit these cases (i.e. not 
replace them with the next randomized case) for three reasons. First, based on my review, such 
events are uncommon, so would not have a material impact on the performance level. Second, 
this part of the protocol could not easily be “gamed”; the vendor would need to either 
inappropriately restart medications, or cause the patient to die, be transferred to a private prison, 
or be released, options which are either unethical (and grounds for revocation of a provider’s 
license), illegal, or beyond their powers. Third, the protocol could be written in a way that would 
exclude such cases. However, such a provision in the protocol would be very complicated. As I 
have already observed in this case, when protocols get that complicated, they open the door to 
gaming. Thus, on balance, including such cases is the most rational and practical solution. 
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Recommendation 31: 

If the results of this retrospective re-audit are substantially Compliant, as defined in the 
Stipulation, PM 86 should be terminated. 

The Plaintiffs concur “except that Plaintiffs do not agree that the reported scores may be 
counted as compliant to the extent that they count as “compliant” encounters lasting 5 
minutes or less.” The Defendants concur and state “Should the Court order Defendants to 
re-audit and the results show compliance, the Defendants agree the measure should 
terminate.” 

Recommendation 32: 

If PM 86 is not terminated, going forward, it should be measured in the following 
manner. For clarity and simplicity, I use the month of July as the audit (CGAR) month. 

Prospective Audit Protocol: 

1. The Source Document is all patients who were listed as being MH-3D (“have 
been recently taken off of psychotropic medications and require follow up to 
ensure stability over time.”) during the month of July. 

2. Randomize the list. 

(Randomization may be done at any point before selection of cases; I have 
arbitrarily placed it as step 2.) 

3. Exclude all patients except those whose date of stoppage or date of a previous 
visit with a mental health clinician was in April. 

(Excluding cases may be done at any point; I have arbitrarily placed it as 
step 3.) 

(If the provider instructed the patient to taper the medication rather than 
stop it abruptly, the date of stoppage is the last date the patient took the 
medication. If the patient stopped the medication some time earlier on 
his/her own, the date of stoppage is the date of the visit with the provider 
when the provider acknowledges and agrees with the stoppage.) 

4. Select the first 15 44 patients in each yard. 


44 The logic of requiring a sample of 15 instead of the usual sample of 10 is as follows. Under the 
current protocol for PM 86, if a patient is transferred out of a yard in the middle of the post 
follow-up period, the patient is excluded from the audit. As a result, the only patients who 
currently remain in the audit are those with a complete follow-up period, which is comprised of 
two follow-up visits. Under the proposed Prospective Protocol, patients who transfer out will no 
longer be excluded. As a result, such patients may only have one follow-up visit to audit. Thus, 
whereas under the current protocol there are always 20 data points in the audit of a yard (10 
patients x two visits each, assuming there are 10 eligible patients in the yard), under the proposed 
protocol, there may be fewer than 20. The Plaintiffs were concerned about this reduction of data. 
To address this issue, ADC examined the frequency with which patients are transferred from a 
yard following discontinuation of psychotropic medications. Based on the results of that 
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(If the patient moved from one yard or facility to another during the 
month, the case should be audited under the yard where the patient resides 
at the end of July, the day the Source Document is produced.) 35 
5. The monitor should test: Was the patient currently safe during the entirety of 
this month? If so, the case is Compliant. In other words, phrased in the negative: 
Was there any day during the month when the patient was still within six months 
of discontinuation of a psychotropic medication but had gone more than 90 days 
without seeing a mental health clinician? This methodology differs from the “X” 
day methodology ordered by the Court in its previous order (Doc. 2225) and thus 
requires vacating of that order. 

(The fact that not all months have 30 days means that some cases may not 
get selected for audit and some cases may be selected for, arguably, the 
wrong month. These events should be rare. Neither “error” is material nor 
would reasonably be expected to lead to biased results. Therefore, in the 
interest of simplicity, this 3-month rule should serve well.) 

The Plaintiffs concur. Defendants “do not agree that past results are unreliable and should 
be re-audited” but did not offer any specific methodological flaw in my analysis. They 
“maintain that they have been substantially compliant with this measure and it should be 
terminated pursuant to the terms of the Stipulation.” 


examination, increasing the sample size from 10 to 15 should compensate for any reduction in 
per-patient data points. 
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Part IA - Retirement, Collapsing, or Modifying Measurement of PMs 

In the following section I recommend the retirement, collapse, or modification of PMs. This is 
responsive, albeit indirectly, to the Court’s instruction in three ways. First, the Court seeks 
recommendations for alleviating causes of, or barriers to, compliance with the PMs. Compliance 
with, and monitoring of, the PMs drives much workload for both ADC and the vendor. To the 
extent that compliance with a given PM does not improve patient safety, continued measurement 
of that PM diverts valuable resources away from achieving compliance with the remaining PMs. 
Thus these non-value-added PMs contribute to non-compliance. Second, the Court seeks 
evidence of how failures to successfully perform on PMs pose a risk of harm. This inquiry, 
addressed in Part II of my report, indirectly begs the question of whether there are PMs, failure 
of which do not post a risk of harm. Third, in Part IV of my report, I will be addressing the 
question “whether the PMs by themselves accurately reflect the adequacy of the care being 
provided to prisoners,” an analysis which the Court states “will assist the Court in tailoring the 
appropriate remedial measures.” In that analysis I will identify gaps in measuring the adequacy 
of the care being provided to prisoners. In the event that that gap analysis leads the Court to 
introduce new PMs, it is only fair to reduce the work load burden of PM monitoring by removing 
less useful or non-useful ones. 

PM 12 ( Medical record will contain documentation of refusals or “no shows. ”) 

Issue: 

This PM is conceptually flawed in two ways. First, it legitimizes a behavior (“no show”) 
that should not be acceptable in a prison environment. Staff should always know where 
patients are and policy should require that either patients present themselves to the locus 
of medication administration or the medical staff arrange for the patient to receive the 
medications in an alternative location (e.g. legal visit, classroom, court). In a prison 
environment, failure of a patient to present for medication administration (or, in fact, any 
other scheduled health-related activity) cannot be assumed to be volitional on the part of 
the patient. Indeed, there can be other reasons such as: COs coerced the patient to no 
show; other residents coerced the patient to no show; the patient is suffering a side effect 
of the medication and is unable to show. 

Second, it encourages a behavior (having the nurse have the patient sign a refusal form) 
that is counterproductive (and unnecessary). It is counterproductive because it messages 
to nurses that when a patient refuses a medication, the nurses’ responsibility to the patient 
has been discharged by signing the form. In fact, for medications where missed (refused) 
doses pose a risk to the patient, follow-up intervention by an RN or provider to try to gain 
the patient’s compliance with taking the needed medication or find an alternative 
medication, is what is needed. And for those medications where missing doses poses little 
risk, signing of a refusal form serves no purpose. 
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Recommendation 33: 

PM 12 should be retired. 

The Parties concur. 

PM 15 ( Inmates who refuse prescribed medication (or no show) will be counseled by a QHCP 
after three consecutive refusals.) 

Issue: 

This PM endorses a staff action which is dangerous or wasteful. The PM is predicated on 
the assumption that missing fewer than three consecutive doses, or missing more than 
three non-consecutive doses, of a medication is safe. In fact, for some medications (e.g. 
insulin, antibiotics, antivirals), missing even a single dose incurs risk. For some 
medications (e.g. anti-seizure medications), missing several non-consecutive doses incurs 
risk. Conversely, for some medications (e.g. ibuprofen), missing three or more 
consecutive doses poses little risk, and therefore counseling is unnecessary and diverts 
valuable staff resources from more valuable tasks (such as tasks which contribute to 
achieving compliance with other PMs). 

Recommendation 34: 

Going forward, PM 15 should be replaced (or reworded) to require that patients who 
refuse prescribed medications will be counselled by a QHCP in accordance with policy. 
Policy should be modified such that for certain missed medications (or classes of 
medications) and dosage-missing pattern, the medication nurse is triggered to escalate the 
case to a higher authority in a specified amount of time. The higher authority is an RN or 
provider who is then responsible for determining the reason for the refusal and securing 
the patient’s adherence with the medication, finding a clinically appropriate alternative 
treatment, or assuring that the patient is making an informed refusal. Exceptions to policy 
should be allowed, on a case-by-case basis, pursuant to a provider’s order. The policy 
should also address repeat occurrences in a way that ensures patient safety, but 
recognizes that once a patient has made an informed refusal to take medications as 
prescribed, it may not be a good use of staff resources to trigger the counseling pathway 
described above. Finally, the algorithm for triggering the case escalation is best 
incorporated into the medication administration software in eOMIS such that the 
medication nurse is automatically alerted when counseling is necessary, rather than 
relying on nurses’ memory during the busy stressful task of medication administration. 

The Plaintiffs concur. The Defendants do not concur for two reasons. First, they state that 
“It will be difficult to identify and agree upon classes of medications [and] an acceptable 
algorithm for a missing dosage pattern.” In crafting this recommendation I purposely 
specified that the medications and missing dosage pattern would be dictated by policy, 
thus putting direct control of these decisions in the hands of the Defendants based on the 
clinical judgment of their medical director and thus allaying concerns about reaching 
agreement. Second, they state that “It will be difficult to.. .have eOMIS trigger automatic 
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responses to different triggers.” The Defendants are likely correct in that eOMIS 
probably does not have this as a pre-set capability and will require programming. I share 
Defendants’ implied concern that such programming will require additional fiscal 
resources. This reinforces my assertion in Part in of this report that ADC health service is 
severely underfunded and that necessary improvements in eOMIS is one of the key areas 
where such funding is needed. 

PM 16 ( Perpetual inventory medication logs will be maintained on each yard.) 

Issue: 

This PM (and PM 18 and PM 19) was relevant to patient safety in a previous era when 
pharmaceuticals were managed in a different way. In the current (and certainly future) 
pharmacy delivery management system, compliance with this PM has no correlation with 
patient safety. It is, instead, a PM which measures internal control of assets. While 
monitoring of inventory may have some business value to the vendor, it has no place in a 
set of measures designed to monitor patient safety. Any small role this PM might have 
served to assure seamless administration of medications is better measured by other 
existing PMs plus ones I recommend elsewhere. 

Recommendation 35: 

PM 16 should be retired. 

The Parties concur. 

PM 18 ( Daily delivery manifests will be kept in binders located in medication rooms on each 
yard/complex and will be reviewed and initialed daily by an LPN or RN.) 

Issue: 

See PM 16 

Recommendation 36: 

PM 18 should be retired. 

The Parties concur. 

PM 19 ( Perpetual inventory medications will be signed off on the Inmate's individual MAR.) 
Issue: 

See PM 16 

Recommendation 37: 

PM 19 should be retired. 

The Parties concur. 
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PM 20 (Medical AIMs entries are accurately completed within 3 business days from the entry in 
the medical record.) 

Issue: 

This PM measures how reliably “SNO” (Special Needs Orders) are transferred from 
eOMIS to AIMS (the electronic management system used by custody to implement 
health-related accommodations that require custody staffs actions, e.g. allowing a patient 
to carry a cane, assigning a patient to a lower bunk, preparing a medical diet in the 
kitchen). It is problematic in a number of ways. First, it is a vestige of the days of paper 
communications. Currently almost all SNO communications are sent from eOMIS to 
AIMS electronically, immediately, and automatically, rendering nearly useless the 
auditing of this activity. Second, the designated Source Document for this PM is a report 
generated by AIMS. Given that the goal of the PM is to identify failures of 
communication from eOMIS to AIMS, using AIMS as the starting point is illogical. 
Third, the only SNO communications which are sometimes (but rarely) not transferred 
from eOMIS to AIMS electronically are orders for medical diets. However, (a) the 
reliability of implementation of medical diets is already measured by PM 71 ( Inmates 
with diagnosed and documented diseases or conditions that necessitate a special diet will 
be provided the diet, if clinically indicated. Wien prescribing the special diet, the 
provider will include the type of diet, duration for which it is to be provided, and any 
special instructions.)', and (b) the fact that a diet order is missing from AIMS does not 
mean it was not executed (medical staff often call the kitchen directly to set up medical 
diets, so the notification may not appear in AIMS), making AIMS an unreliable Source 
Document. Fourth, until just recently, this PM was audited by multiple auditors who 
applied different thresholds for compliance. For example, when I reviewed the October 
2018 CGARs for the Florence Complex - a month in which they were found 
Noncompliant - audit by another monitor who adhered more strictly to the letter of the 
standard would have resulted in a finding of Compliant. Fifth, auditing of this PM is 
highly work-intensive relative to other PMs. It uses resources that could be re-invested in 
more meaningful measures which are more key to reducing risks to patient safety. 

Recommendation 38: 

PM 20 should be retired. 

The Parties concur. 

PM 32 (A final independent clinical mortality review will be completed by the Health Services 

Contract Monitoring Bureau for all mortalities within 10 business days of receipt of the medical 

examiner’s findings.) 

Issue: 

The clinical mortality review (MR) is an important process because it identifies problems 
requiring remediation by the vendor. Many of these problems pose a significant risk of 
serious harm. As such, whether or not they contributed to the death in the case under 
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review, failure to remediate the problem poses a risk to future patients. Thus it is 
necessary to identify and remediate the problem as quickly as possible. 

As currently phrased, PM 32 requires ADC to wait until after receipt of the medical 
examiner’s report to complete the MR. Unfortunately, medical examiner reports can be 
delayed for extended periods of time. For example, Patient 21 died in May, 2017, but the 
medical examiner did not produce a report until January, 2019. Any “lessons learned” 
through the MR process would thus have had resultant remediations implemented almost 
two years late. 

While the medical examiner’s report often provides some illumination about the 
mechanism and cause of death, based on my experience, my review of ADC MRs, and 
discussion with ADC’s MR Committee, it almost never changes the analysis of problems 
requiring remediation by the vendor. Indeed, the MR process is not a tort-related process 
to find a proximate cause of damage, but rather to identify and fix dangerous conditions, 
regardless of the outcome in a single case. The lens through which the process examines 
the case is: what did people know and rely upon at the time the patient was cared for. It is 
for this reason that the post hoc information provided by the medical examiner so 
infrequently changes the analysis. Thus waiting for the medical examiner’s report allows 
dangerous conditions to remain unaddressed too long. 

Recommendation 39: 

Going forward, PM 32 should be modified as follows: “An independent clinical mortality 
review will be completed by the Health Services Contract Monitoring Bureau for all 
mortalities within one month of the death. Within 10 business days of receipt of the 
medical examiner’s findings, the Health Services Monitoring Bureau will affix an 
amendment to the mortality review. The amendment will contain any additions, deletions, 
or modifications necessitated by the information contained in the medical examiner’s 
report, or will state that its original findings and recommendations stand as is.” 

The Plaintiffs concur. The Defendants agree with the recommendation to conduct a 
mortality review within a month of the death, but do not wish to change the wording of 
the PM which “would require a change in the terms of the Stipulation” and want the PM 
to be measured by use of the final (post-medical examiner-report) mortality review. As 
long as a review with attendant recommendations for improvement is conducted within 
one month based on the information available at that time, so that efforts toward 
improvement are not delayed until receipt of a medical examiner’s report, the 
Defendants’ comments are consistent with the intent of this recommendation. 
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PM 35 (All inmate medications (KOP and DOT) will be transferred with and provided to the 
inmate or otherwise provided at the receiving prison without interruption.) 

Issue: 

The effectiveness with which medications are given, without interruption, across a 
transfer from one complex to another, is dependent on the health care staff at both the 
sending and receiving complexes doing their jobs. However, the lion’s share of that 
responsibility (and the most common source of errors) lies with the sending complex. 
However, as currently designed and reported, this PM measures and penalizes, 
respectively, the receiving complex. This happens because the Source Document is 
drawn from patients who arrive at the receiving complex, and the errors are then ascribed 
to the receiving complex. As a result, it is difficult to provide appropriate feedback to the 
sending complex, which in turn is a barrier to ever improving performance on the PM. 

Recommendation 40: 

Going forward, the sample for PM 35 for a given month at all complexes should either be 
drawn from patients transferring into each complex with the resultant score attributed to 
the receiving complex (i.e. as is currently being done) or drawn from patients transferring 
from each complex, with the resultant score attributed to the sending complex. The 
method for a particular month should be chosen randomly. 45 

The Plaintiffs concur. The Defendants do not concur, despite the modification explained 
in the footnote, stating “The responsibility should remain with the receiving facility. The 
final action of assuring the inmate has his medications rests with the receiving facility. If 
the medication is not sent with the inmate, the receiving facility can assure compliance by 
using clinic stock or back-up pharmacy.” 


45 My original recommendation modified PM 35 such that audit cases would be drawn 
exclusively from patients transferring from each complex. In their comments to the draft of this 
report, Defendants disagreed with that approach, explaining, “Better patient care lies with the 
onus on the receiving complex to ensure that the patient has required medications. Placing the 
responsibility on the sending complex does not incentivize them to follow up and ensure an 
inmate immediately receives medication at the receiving complex after it was inadvertently 
missed in transfer. There is also no incentive for purposes of compliance with this measure for 
the receiving complex to ensure the incoming inmate receives required medications 
immediately.” Defendants reasoning is sound. Measuring only the performance of sending 
complexes could have the unintended consequence of deterioration of performance on the 
receiving side. Because there is still value to incentivizing proper performance by the sending 
complex, the best approach to improve performance on PM 35 is to incentivize both the sending 
and receiving complexes. To avoid burdening Defendants with additional workload, I therefore 
redesigned the recommendation such that ADC would only need to use one method each month, 
not two. By alternating randomly between methods, the vendor will not be able to predict which 
method will be used in a given month. Thus both the sending and receiving complexes will be 
incentivized every month to assure that patients receive their medications seamlessly when 
transferred. 
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PM 42 (A follow-up sick call encounter will occur within the time frame specified by the Medical 
or Mental Health Provider.) 

Issue: 

This PM is difficult to monitor due to the paucity of events which meet the parameters of 
the PM. One main obstacle is that the Source Document for the sample is drawn from 
events during the previous month. Depending on when the follow-up is ordered for and 
when in the month the audit is conducted, the follow-up event may be in the future, and 
thus the case must be skipped. Thus monitors pore through numerous randomized 
potential cases to find one which is auditable. In the past this has led some monitors to 
look backwards in time for events to monitor, which was stopped by the Court. The PM 
also fails to fully achieve the original goal of the PM which was to examine whether, 
when a physician issues patient care orders, e.g. wound care, that the order is carried out. 
This happens because the current protocol required for PM 42 specifies that the patient 
must have first been seen in a providers’ clinic, and many relevant orders are generated 
from other encounters. 

Recommendation 41: 

Going forward, PM 42 would be re-worded to read, “After a visit with a Medical or 
Mental Health provider, any follow-up encounter with a provider (excluding chronic care 
visits) or nurse will occur within the time frame specified by the provider.” The audit 
procedure would be modified as follows: 

1. The Source Document is a list of all follow-up appointment scheduled to take 
place in the month under audit. 

(By including all scheduled appointments, rather than all completed 
appointments, appointments that were scheduled but never took place will also be 
audited.) 

2. If the date of the follow-up appointment is no later than the date the 
appointment was ordered, the case is Compliant. 

(This will require a programming change to eOMIS such that when providers 
order follow-up appointments, they must also choose a date by which the 
appointment must take place.) 

The Plaintiffs concur. The Defendants do not concur, stating they “object to a change in 
wording within PM 42. It is a change in the Stipulation.” 

PM 64 (In an IPC, a Medical Provider evaluation and plan will occur within the next business 

day after admission.) 

and 

PM 65 (In an IPC, a written history and physical examination will be completed by a medical 
provider within 72 hours of admission.) 

Issue: 

These PMs both measure the extent to which a provider assesses a new patient upon 
admission to an IPC. PM 64 requires an “evaluation and plan” within one business day. 
PM 65 requires a “history and physical” within 72 hours. First, there is no meaningful 
distinction between these two activities. In the setting of an admission to an IPC, the goal 
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of both is to fully evaluate the condition for which the patient has been admitted to the 
IPC, assure that the IPC is in the proper and safe venue for patient management, and 
establish a plan for further testing, treatment, and monitoring. These activities must be 
completed as soon as possible after admission to the IPC. 

Recommendation 42: 

Going forward, PM 65 should be collapsed into PM 64. 

The Plaintiffs concur. The Defendants do not concur, stating “Defendants want PM 64 and PM 
65 to remain the same without collapsing PM 65 into PM 64. Defendants believe the 
recommendation requires a change in the Stipulation.” 

PM 66 (In an IPC, a Medical Provider encounters will occur at a minimum every 72 hours.) 
Issue: 

The requirement for a patient in the IPC to be seen every 72 hours is potentially too 
lenient or too strict. It is too lenient in that some patients are ill enough to require more 
frequent visits. It is too strict in that some patients, especially those in the IPC for long¬ 
term stays, are very stable and do not require such frequent visits. Both situations pose a 
risk to patient care, the former because the patient does not get the care he or she needs, 
and the latter because the PM drives unnecessary use of scarce and valuable resource 
(IPC provider), diverting their efforts from patients who truly require their attention. 

Recommendation 43: 

Going forward, PM 66 should require: 

1. A provider sets and documents the frequency of provider visits at the time of 
the provider’s initial evaluation and plan and periodically during the admission as 
necessary. 

(If no frequency is stated, the frequency is no less frequently than every 72 hours.) 

2. That frequency is clinically appropriate based on the patient’s clinical condition 
as determined by the monitor in collaboration with the Monitoring Bureau 
physician. 

(If the frequency is less than every three days, the Monitoring Bureau physician 
must review this case.) 

3. The patient’s medical record contains sufficient clinical data to determine 
whether the frequency of provider visits is clinically appropriate. 

4. A provider examines the patient in compliance with the set frequency. 

5. The patient must be seen by a provider at the set frequency (or more often) for 
each visit until the patient is discharged or the end of audit month, whichever is 
sooner, for the case to be Compliant - there is no partial credit. 

Monitoring of this PM would need to be performed by, or in collaboration with, a provider. 
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The Parties concur. Defendants do not concur because the “change in protocol from an 
objective standard agreed to by the parties to a subjective standard [] is vulnerable to 
disagreement of medical opinions.” 

PM 67 (In an IPC, Registered nurses will conduct and document an assessment at least once 
every shift. Graveyard shift assessments can be welfare checks.) 

Issue: 

Because some nursing shifts are 12 hours long, nurses can be in compliance with this 
PM, yet see the patient at inappropriate intervals. For example, Patient 20 was admitted 
to the IPC on 12/28/18 after having been released from the hospital for heart failure and 
low blood pressure. According to the nursing care plan, he was at risk for going back into 
heart failure and required monitoring of his fluids and lungs. The nurse on graveyard shift 
conducted a “welfare check” on 1/1/19 at 19:00. The nurse on day shift conducted the 
next assessment on 1/2/19 at 17:05. The nurse on graveyard shift conducted the next 
“welfare check” on 1/2/19 at 20:00. Thus the patient went almost a full day (22 hours) 
without any evaluation between 1/1 and 1/2. 

An additional concern is that a “welfare check” requires only a very cursory assessment. 
While nurses often do more, the check for Patient 20 on 1/13/19 at 06:21 consisted of 
only noting that the patient was asleep, did not appear in distress, and his breathing 
appeared normal, which is compliant with the PM. However, given the patient’s 
condition and nursing care plan, he required a deeper assessment, one at the same depth 
as those done during the day shift. 

Recommendation 44: 

Going forward, PM 67 should require that: 

1. RNs conduct an assessment at a frequency in accordance with the nursing care 
plan established by the RN at admission. If the assessment is conducted by an 
LPN, it is Noncompliant. 

2. If the frequency is not specified in the nursing care plan, it is at least twice 
daily. 

3. The patient must be assessed by an RN at the set frequency (or more often) for 
each visit until the patient is discharged or the end of the audit month, whichever 
is sooner, for the case to be Compliant - there is no partial credit. 

(The spacing of the assessments are reasonable as determined by the monitor (an 
RN or higher). For example, if the assessments are supposed to be done every 12- 
hour shift, the monitor would likely determine that an assessment done 10 
minutes before one shift ends and 10 minutes after the next shift begins is 
Noncompliant. Conversely, if the assessments are supposed to done every four 
hours, the monitor would likely determine that an assessment done at 4 hours and 
10 minutes after the previous assessment is Compliant.) 
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The Plaintiffs concur. Defendants do not concur because the “change in protocol from an 
objective standard agreed to by the parties to a subjective standard [] is vulnerable to 
disagreement of medical opinions.” 

PM 72 ( Inmates who refuse prescribed diets for more than 3 consecutive days will receive 
follow-up nutritional counseling by a qualified health care provider.) 

Issue: 

My review of 15 individual cases 46 which were found Noncompliant revealed that none 
of failures to counsel (or delayed counseling, which was the problem in most of the 
cases) posed a significant risk of harm. Most of the cases involved diets for conditions for 
which refusal to follow the diet would be immediately obvious to the patient, and 
therefore continued noncompliance was clearly a personal choice. 47 With one exception, 
in the remaining cases the prescribed diet was clinically unnecessary. In that one 
exceptional case, the diet was clinically necessary and refusal posed a risk. However, the 
patient had been counseled on this issue two months earlier, so was able to make an 
informed decision in his own best interest to refuse the diet. Thus this PM is too broad in 
its specification and may overstate the current risk to patients at ADC. 

Recommendation 45: 

Going forward, the sample for PM 72 should be drawn from only those patients on a 
medical diet for a metabolic disease ( e.g. diabetes, heart disease). Patients who have 
been counseled within the past year for the same diet should be considered Compliant. 
Finally, the PM should specify how soon the counseling must be performed; I 
recommend specifying that counseling be performed within two weeks of the third 
consecutive day of refused meals. 

The Parties concur. 

PM 73 (MH-3A prisoners shall be seen a minimum of every 30 days by a mental health 

clinician.) 

and 

PM 77 ( Mental health treatment plans shall be updated a minimum of every 90 days for MH-3A, 
MH-4, and MH-5 prisoners, and a minimum of every 12 months for all other MH-3 prisoners.) 
and 

PM 80 (MH-3A prisoners shall be seen a minimum of every 30 days by a mental health 
clinician.) 


46 For this review, I chose the Eyman complex, because it was the only one which was 
Noncompliant in the December, 2018 CGAR, the default CGAR used for most of my tests. I 
reviewed all 15 cases that were found Noncompliant. 

47 An example of this is a lactose-free diet for a patient who is presumably lactose intolerant. A 
patient who chooses to ignore this diet would soon be aware of the consequences of his or her 
choice - cramps, flatus, or diarrhea - and have the information he or she needs to decide whether 
to comply with the diet. 
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and 

PM 82 (MH-3B prisoners shall be seen a minimum of every 90 days by a mental health 

clinician.) 

and 

PM 87 ( MH-4 prisoners shall be seen by a mental health clinician for a 1:1 session a minimum 
of every 30 days.) 

Issue: 

Patients in the MH-3 and MH-4 groups are the most stable patients with current mental 
illness diagnoses. These five PMs, which require one-to-one counseling, were created to 
ensure access to mental health care for these (and, for PM 77, MH-5) patients. 
Implementation of these PMs has reportedly been very effective in improving access to 
one-to-one care. However, it has turned out to have an unintended consequence which 
materially erodes some of that improved access to care. 

There are a number of treatment modalities in the armamentarium for treating mental 
illness, such as medications, individual counseling, and group therapy, among others. The 
most appropriate treatment for a given patient is one or more of these modalities, each at 
a certain “dose.” One patient with mental illness X may need medications and one-to-one 
counseling, whereas another patient with the same illness may need group therapy. The 
shortcoming of these five PMs is the presumption that one-to-one counseling is necessary 
for all patients. A shortcoming of the mental health-related PMs globally is that, with the 
exception of PM 92 {MH-3 and above prisoners who are housed in maximum custody 
shall be seen by a mental health clinician for a 1:1 or group session a minimum of every 
30 days.), they attach no value to provision of group therapy. This concept was reinforced 
by the Court when it clarified measurement of PMs: “.. .group counseling does not count 
toward compliance in any of the other [other than PM 92] Performance Measures.” (Doc. 
1673 at 5) In response to the requirements of these five PMs (and in the absence of more 
than 1 PM valuing group therapy), mental health clinicians spend considerable time 
seeing patients for one-to-one sessions. Many of these sessions are not clinically 
indicated (the clinicians do not find them necessary and/or the patients do not want 
them). The time spent on these non-value-added sessions is an important factor 48 
impairing clinicians from having time to provide an adequate number and variety of other 
treatment modalities, such as group therapies. Such a mental health management system 
is not consistent with mental health treatment in the community: patients with MH-3- or 
MH-4-level disease who have decision-making capacity receive one-to-one treatment 
only when the clinician and the patient agree it is beneficial. Moreover, the failure to 
provide group therapy when needed poses a significant risk of serious harm to patients. 

Recommendation 46: 


48 Insufficient staffing levels, at least during the Corizon contract, is another important factor. 
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Going forward the MH-3- and MH-4-related portions of PMs 73, 77, 80, 82, and 87 
should be collapsed into a single PM which requires that: 

1. Patients receive one-to-one counseling from a mental health clinician and/or 
group therapy conducted by a qualified mental health professional at a frequency 
consistent with their clinical need, established with input from the patient. 

(While correctional officers may conduct any appropriate group therapies, such 
therapies will not count toward compliance with this PM.) 

2. Those frequencies should be established and documented by a clinician who is 
licensed in the State of Arizona, within 30 days of arrival or of designation as a 
MH-3 or MH-4. 

3. The frequencies should be modified (and documented) periodically by a 
clinician who is licensed to practice in the State of Arizona, as clinically 
indicated. 

4. All patients in these categories must receive one-to-one counseling from a 
mental health clinician at least once a year. 

Monitoring of this collapsed PM would need to be performed in collaboration with a 
mental health clinician (licensed to practice) or provider (provider). The MH-5-related 
portion of PM 77 remains as is. 

This modification also requires vacating of the Court’s previous order prohibiting the use 
of group therapy for purposes of complying with these PMs. (Doc. 1673 at 5) 

Neither Party concurs entirely. Plaintiffs “do not object to this revision in principle, but it cannot 
function to provide minimally adequate care as long as treatment plans ‘reveal a marked lack of 
comprehensiveness’ and ‘are generally inadequate,’ as described elsewhere in this report.” 
Plaintiffs’ concern is noted and accurately reflects problems described elsewhere in the report. 
However, these PMs, as originally constructed, address only frequency of care, not quality of 
care. The proposed modification is not different, but makes the frequency more clinically 
relevant. Defendants do not concur because my “recommended change in protocol from an 
objective standard agreed to by the parties to a subjective standard [] is vulnerable to 
disagreement among medical opinions.” Defendants do agree with the recommendation that 
group therapy be allowed. 


PMs 73, 77, 80-84, 87-89, 90, and 92 (For brevity , I have not reprised each of the PM texts.) 

Issue: 

This group of PMs are mental-health related and contain a common element: a 
requirement for a visit, on an unlimited recurring basis, with a clinician or provider. 
Discussion of the appropriate methodology for auditing these PMs has consumed an 
enormous amount of the Parties’ and Court’s time (See discussion of “Seeing Patients 
Every ‘X’ Days” in Part I of this report.). While conceptually simple on the surface, as 
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the Court record shows, operationalizing the concept into a measurable PM is not that 
simple. For example, as Plaintiffs have pointed out, if a visit which is supposed to occur 
within three months is audited during month 2, the case cannot possibly be found 
Noncompliant. On the other hand, excluding it from audit fails to give Defendants 
“credit” for having conducted a visit early. There are other special circumstances, the 
audit options of each of which has its tradeoffs. 

In my Recommendation, I propose a simple methodology based on a medical model of 
vaccination. When we administer a vaccine to a patient, we are giving them a dose of 
treatment that keeps them “safe” until the next dose is due. If we were going to design a 
PM to measure vaccination, for, say, tetanus (which requires a vaccination every 10 
years), we would draw a random sample of all patients and check to make sure that each 
one is protected at the time of the sampling. Whether they received their last vaccination 
last year and are not yet due for their next shot, or received it nine years ago and are due 
this year would be irrelevant. The only question would be: is the patient currently safe by 
virtue of having a recent enough vaccination? By analogy, mental health visits can be 
viewed as “vaccinations” - treatments to keep the patient safe until the next required 
visit. It is therefore very reasonable to audit mental health visits by asking the same 
question: is the patient currently safe? 

Recommendation 47: 

PMs 73, 77, 80-84, 87-89, 90, and 92 (to the extent they are not collapsed or retired, as 
recommended elsewhere in this report) should be audited by the following methodology 
going forward. The sample for the month in question should be drawn from all patients 
defined in the PM. For example, for the January CGAR, the sample for PM 73 {All MH-3 
minor prisoners shall be seen by a licensed mental health clinician a minimum of every 
30 days.) would consist of all minors who are MH-3 in January and have been MH-3 for 
30 days or more. The monitor should test: Was the patient currently safe during the 
entirety of this month? In other words, phrased in the negative: Was there any day during 
the month when the patient was out of compliance with the timeframe of all necessary 49 
treatments? This methodology differs from the “X” day methodology ordered by the 
Court in its previous order (Doc. 2225) and thus requires vacating of that order. 

Neither Party concurs entirely. Plaintiffs “do not object in principle,” but again express 
concern that the quality of care during an encounter is adequate: “.. .this 
Recommendation requires further elaboration of the methodology for determining 
whether a given record is compliant or noncompliant And also state “The proposed 


49 In the current PMs, necessity is defined via fixed intervals for repeat visits, e.g. “... .no less 
than every 90 days.” In the previous recommendation I recommend modifying some PMs such 
that the interval is variable, determined by the professional and the patient, on a case-by-case 
basis. Should those other recommendations not be adopted, my recommendation here is worded 
so that it would work in either case - fixed or variable intervals. 
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modification needs to explain, concretely and for each PM, how the monitor would 
determine if the patient was “safe.” As in the previous recommendation, I am only 
proposing exchanging one way of measuring the timing of visits with another and not 
changing the way quality of care is evaluated. Defendants do not concur because my 
“recommended change in protocol from an objective standard agreed to by the parties to 
a subjective standard [] is vulnerable to disagreement among medical opinions.” Their 
comment is understood by referring to my previous recommendation wherein I propose 
using variable (requiring clinical judgment), rather than fixed intervals between visit. As 
noted in the footnote, the current recommendation is workable with fixed or variable 
intervals between appointments. If the Court does not accept the previous 
recommendation (i.e. instructs the Parties to maintain fixed intervals), then the 
Defendants’ concern about subjectivity would be moot. 

PM 94 ( All prisoners on a suicide or mental health watch shall be seen daily by a licensed 
mental health clinician or, on weekends or holidays, by a registered nurse.) 

Issue: 

The court-ordered procedure for auditing of this measure requires monitors to exclude 
(skip) any patient who is transferred to another yard during the watch. In my opinion, 
excluding such cases is wrong for two reasons. First, patients who are transferred during 
the middle of a watch are likely to be different in some systematic way from those who 
are not transferred, a difference that may be reflective of the fact that the patient is having 
some difficulty at the first yard. Such patients are in greater, not lesser, need of 
monitoring. Second, transitions from one yard to another are events at high risk for 
breaks in continuity of care. In other words, it is across such transitions that the need for 
follow-up may be lost. Therefore it is particularly important to monitor these patients. 

Recommendation 48: 

Going forward, PM 94 should not exclude patients who are transferred to another yard 
during their watch. Neither the beginning nor the end of their watch should be excluded. 
In other words, if a patient starts his or her watch at yard A and is transferred to yard B 
during the watch, the watch event should be eligible for random selection on the Source 
Document for yard A as well as the Source Document for yard B. If that patient is 
randomly selected for audit at yard A, the monitor will test for compliance with the PM 
for all days the patient was at yard A during the month under review; if selected for 
sampling for yard B, the monitor will test for compliance with the PM for all days the 
patient was at yard B during the month under review. 50 

50 By including truncated cases (cases which are missing days at the back end of the watch 
(“right-censored”), e.g. yard A, or days at the front end of the watch (“left-censored”), e.g. yard 
B), the total number of days being measured will decrease a little. Because ADC rarely transfers 
patients in the middle of a watch, and because, even ignoring the loss of data for days after the 
transfer, there are still a plentiful number of data point for days prior to the transfer, in my 
opinion, the reduction in statistical power of the PM from this loss is likely to be negligible, and 
is greatly outweighed by the value of making the measure more meaningful and accurate. 
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As noted in Recommendation 2,1 recommend that the unit of analysis for this PM should 
be the event, not the patient. As such, a patient who was placed on watch twice during the 
audited month might have both of his or her watches included. 

The Parties concur. 

PM 95 ( Only licensed mental health staff may remove a prisoner from a suicide or mental health 
watch. Any prisoner discontinued from a suicide or mental health watch shall be seen by a 
mental health provider, mental health clinician, or psychiatric registered nurse between 24 and 
72 hours after discontinuation, between seven and ten days after discontinuation, and between 
21 and 24 days after discontinuation of the watch.) 

Issue 1: 

PM 95 suffers from the same problem described above in PM 94, i.e., that cases 
randomly selected for audit as required by the Court are skipped and replaced if the 
patient is moved to a different yard during the 24-day observation period after removal 
from watch. The provenance of this procedure is clearer than for PM 94: the procedure 
was instructed by the Court at the request of Plaintiffs. (Transcript of Status Hearing, 
11/7/17, discussion beginning at 163, instruction at 169) The rationale for the procedure 
was that PM 95 tests for compliance at four junctures (removal from watch by a licensed 
staff on Day 1 and 3 follow-up visits over the course of the next 23 days) and so 
including a patient who was transferred during the 24-day would truncate some of the 
data, i.e., the performance level would be based on fewer data points. For the same 
reasons described for PM 94 above, excluding patients who are transferred is 
scientifically unwise. 

Issue 2: 

PM 95 suffers from an additional problem: a randomly selected case is excluded if the 
patient is placed back on watch prior to completion of the 24 day monitoring period. In 
my opinion, this exclusion is scientifically wrong. If a patient who was on a watch has to 
be placed back on watch after a short period of time, that may be an indication that their 
follow-up care was somehow deficient. Thus, monitoring such patients for compliance 
with follow-up care is of greater, not lesser, importance than monitoring other patients. 

Recommendation 49: 

Going forward, the following changes should be made to the protocol for auditing PM 95. 
The Court’s previous instruction of 11/7/17 should be reversed: when drawing random 
patients at a yard for PM 95, the patient should not be excluded if he/she transferred into 
or out of the yard during the 24-day follow-up period after discontinuation of watch. 
Neither the beginning nor the end of their follow-up period should be excluded. In other 
words, if a patient is removed from watch and starts his or her follow-up period at yard A 
and is transferred to yard B later during the follow-up period, the follow-up period should 
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be eligible for random selection on the Source Document for yard A as well as the Source 
Document for yard B. 

If that patient is randomly selected for audit at yard A during the month under review, 
he/she should be included in the audit up to the time when he/she departed yard A; 
conversely, if the patient is randomly selected for audit of at yard B during the month 
under review, he/she should be included in the audit from the time he/she arrived at yard 
B until the 24-day watch period ends. (In some cases, a patient might be randomly 
selected for audit at yard A and B.) Further, if a patient’s follow-up period is cut short by 
re-placement on watch, the shortened period should still be audited if randomly selected. 
Finally, unlike for PM 94 (see footnote 50 above), transfer of a patient in the middle of 
the post-watch follow-up is not as uncommon. Therefore, the audit sample for PM 95 
should be increased from 10 to 12 cases per yard to make up for the lost data from 
including truncated cases. 51 

As noted in Recommendation 2,1 recommend that the unit of analysis for this PM should 
be the event, not the patient. As such, a patient who was placed on watch twice during the 
audited month might have both of his or her post-watches periods included. 

The Parties concur. 


51 At my request, ADC conducted a review of the frequency of transfers of patients during the 
24-day post-watch follow-up period. For the month reviewed, of 758 patients on watch, 87 
(11%) were transferred. Increasing the sample size by 20% (from 10 to 12) should more than 
compensate for any data loss because: (a) A compensatory increase of 11% in the sample size 
would assume that all four data points are lost for each transferred (truncated) case. In fact only 
1, 2, or 3 data points are lost when including transferred patients, so an 11% increase would more 
than compensate for the loss.; (b) The previous point tells that even an 11% increase in sample 
size would more than make up for lost data. Rounding up from 11% to a 20% sample size further 
assures that the revised protocol will generate more, not less, data than the current protocol. 
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Part IB - Termination of PMs 

Termination of PMs in the Face of Fewer than 24 Months of Data 

Issue: 

There are months during which no auditable events occur at some complexes for some 
PMs. These months are marked as “N/A” (no data available, not to be confused with 
“N/A” which is also used in Parsons documents to denote “not applicable”) in the 
CGARs. The Court opined that N/A does “not count either for or against termination” 
and that therefore “the lookback period [to determine when a PM could be terminated] 
would be extended to capture 24 months 52 of data.” (Doc. 2900 at 4, 6/22/18) While this 
approach made sense at the time, with another year of data behind us, it deserves 
reconsideration. The practical implication of the Order is that a PM might not be eligible 
for termination for several years. For example, over the past four years (49 months) at 
Douglas, PM 25 {A first responder trained in Basic Life Support responds and adequately 
provides care within three minutes of an emergency .) has been compliant at the 100% 
level for five months. For all the other 44 months, no data was available due to the 
paucity of medical emergencies at the complex. If the current pattern at Douglas 
continues (and assuming the same satisfactory performance level), Douglas would be 
eligible to have PM 25 terminated in the Fall of 2034. This is not an isolated example. All 
told, 62 complex/PM pairs have not accumulated 24 months of non-“N/A” performance 
levels as of March, 2019 (see Exhibit 1). On average, these 63 pairs have accumulated 11 
months of performance levels (less than half of what they need) over the past 49 months, 
with a range of 0 to 23 months. Thus some complex/PM pairs are likely to accumulate 
the requisite 24 months of data within the next month or two, whereas some might never 
accumulate the requisite data to settle the case. 

The reason for the dearth of performance levels is that the PMs in question measure tasks 
provided to sicker patients or measure complications of health diseases or provision of 
health care, and healthier less complicated patients are usually placed in these complexes. 
Theoretically, for a few of the PMs, the lack of data may be - in part, at least - a proxy 
for better care (e.g. PM 30: The initial mortality review of an inmate’s death will be 
completed within 10 working days of death.). 

Any decision about this issue should be informed by the actual performance levels in this 
group of complex/PM pairs. During the most recent 34 months, there were over 2100 
complex/PM-months for these pairs. Of the approximately 500 non-“N/A” months, 452 
were 100% compliant, about 10 were Compliant below the 100% level, and 20 (1%) were 
Noncompliant. Thus when there is data, compliance is very high (99% of months). 


52 The 24-month requirement harkens back to the terms of the Stipulation, wherein a PM at a 
complex is eligible for termination if it has had no more than six Noncompliant months in 
previous 24 months (and no more than two consecutive Noncompliant months in the previous 18 
months). (Doc. 1185 at 4) 
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Finally, in considering ways to approach this issue, one should consider the goals and 
science of monitoring. Safety monitoring should focus on events which are either high 
risk or high frequency, and the amount of monitoring should mirror these two factors. 
Thus to the extent that the dearth of data tells us that the events are of low frequency, in a 
rational monitoring system one should decrease the intensity of monitoring. From a 
mathematical standpoint, the performance levels reported for this group should be viewed 
with caution. Whereas performance levels for many other PMs monitored under this 
Stipulation are based on dozens, if not hundreds, of events, the performance levels 
calculated in the months when there are measurable events in this group, are almost 
always based on very small numbers (usually one, two, or three events). Scores based on 
such small numbers are statistically unreliable. 

Recommendation 50: 

There is no perfect or mathematical solution to this issue. But I believe a rational and 
reasonable solution exists. I recommend that ADC continue to measure PMs at 
complexes for which 24 months of data are not available. This is not an onerous task 
given that, on an average month over the past two years, 49 of the 69 PMs have had no 
data to audit, and of the remaining 13 PMs, most have very few events to audit. As an 
overview, the protocol I recommend below seeks to find a fair balance between not 
giving “credit” to ADC when data is absent, but not measuring a low yield PM ad 
infinitum. 

Each of the possible outcomes of continued auditing should be handled as follows: 

1. If 24 months of data become available, and the performance levels satisfy what 
I will propose as modified Stipulation requirements for these PMs, viz. no more 
than six Noncompliant months in previous 24 months which have data , and no 
more than two consecutive Noncompliant months in the previous 18 months 
which have data , then the PM for that complex should be terminated. 

2. If 24 months of data do not become available for these PMs by the time all 
other PMs (i.e., those PMs not listed in Exhibit 1) satisfy the requirements of the 
Stipulation, but considering the months for which there is data, the performance 
levels for these PMs do not violate the modified Stipulation requirements 
proposed above, then the Stipulation should be eligible for closure. In other 
words, dearth of data for the PMs listed in Exhibit 1 should not prevent closure of 
this case. 

3. If the requirements of the Stipulation are satisfied by all other PMs (i.e., those 
PMs not listed in Exhibit 1), but the performance level of any PM listed in Exhibit 
1 violates the modified Stipulation requirements proposed above, then the Court 
should consider each such PM on a case-by-case basis. The reason for this 
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approach is that, as explained earlier, for the PMs listed in Exhibit 1, even if there 
is a month with actual data, that data tends to be based on very few patients. From 
a statistical standpoint, results based on very few patients are unreliable. Thus if 
the performance levels for Noncompliant month(s) are based on audit of very few 
patients, I would advise the Court to give those Noncompliant months less weight 
(and, for example, possibly allow the case to settle), whereas if the Noncompliant 
month(s) are based on audit of many cases, I would advise the Court to give those 
Noncompliant months more weight (and, for example, possibly not allow the PM 
to terminate, and instruct ADC to continue to collect data on those PMs for a 
prescribed period of time). 

The Plaintiffs concur. Defendants state they do not concur, explaining, “Defendants 
should not be penalized for PMs that are not failing. Each of the PMs should be 
considered on a case-by-case basis. Safford has less than 24 months of scores but has 
only been found noncompliant one month since the beginning of monitoring, in August 
of 2015. For the same reasons recommended by Dr. Stern to terminate the other 
complexes with mostly NA scores, this PM at this complex should be terminated.” I 
believe the recommendation is consistent with the Defendants’ wishes: any data-lacking 
PM that would prevent settlement of the case when all other PMs have earned 
Compliance, would be dealt with on a case-by-case basis. 
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Part II - Evidence of “how any failure to successfully perform on PMs poses a significant 
risk of serious harm to patients” (Doc. 3231 at 1) 

For this Part, I focused on PMs at facilities that showed Non-compliance as of February, 2019.1 
drew examples from two sources. Primarily I drew cases from the December 2018 CGAR report 
for relevant complexes, though I drew some cases from other CGAR where necessary 53 .1 also 
used some cases cited by Plaintiffs in Advocacy Letters to the Defendants. The use of these latter 
cases does not introduce bias in that: (a) responsiveness to the Court’s request requires examples, 
and the source of a valid example should be irrelevant, and (b) I only included a case cited by 
Plaintiffs after independently verifying facts in the patient’s medical record. In each of the cases 
I have chosen as examples below, whether I specifically state it or not, the failure to comply with 
the particular performance measure posed a-significant risk of serious harm to that particular 
inmate. However, the failure to comply does not necessarily mean that other patients in different 
circumstances would have been exposed to a significant risk of serious harm, and not all 
individual patient cases at facilities that fail to successfully perform on PMs carry this same level 
of risk. Further, no harm may have actually resulted in the cases I cite: my assessment was 
focused on whether the failure posed a risk of serious harm. I do not opine as to whether any 
particular inmate’s constitutional rights were violated. 

In their comments on the draft of this report, Defendants have made a similar observation 
regarding several of the PMs I discuss in Part II. For clarity, I have looped them all here (PM 5, 
PM 10, PM 11, PM 15, PM 23, PM 24, PM 31, PM 43, PM 45, PM 59, PM 63, PM 64, PM 65, 
PM 68, PM 94, and PM 95) and address them together. They correctly note that for the PMs in 
question, very few facilities remain Noncompliant, and often there have been no Noncompliant 
months at any complex for some time. They therefore assert that there is no systematic failure. I 
interpreted the Court’s instruction for this part of the report to apply more broadly to PMs where 
at least one facility is not yet fully Compliant and did not view my charge to include an opinion 
on the severity of the non-compliance or its systematic pervasiveness, but rather, simply, to 
answer the question of whether there evidence that failure to successfully perform poses a 
significant risk of harm. My inclusion of a PM in this part of the report therefore should not be 
read as more than that. Some failures may pose more risk than others. For example, with the 
exception of February and March of 2018, no facility has been Noncompliant on PM 5 ( Medical 
Records will be accurate, chronologically maintained, and scanned or filed in the patient’s chart 
within two business days, with all documents filed in their designated location.) since November 
2017, whereas half the facilities have abysmal performance as recently as March 2019 on PM 24 
(Emergency medical response bags are checked daily, inventoried monthly, and contain all 
required essential items.) 


53 For example, if I needed to examine cases deemed Noncompliant at a particular complex, but 
that complex had 100% compliance in the December 2018 CGAR, I looked at CGAR results for 
a month in which the complex’s compliance was less than 100%. 
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PM 5 (Medical Records will be accurate, chronologically maintained, and scanned or filed in 
the patient’s chart within two business days, with all documents filed in their designated 
location.) 

Patient 14 has multiple health problems, including heart disease, hyperthyroidism, hypertension, 
deep vein thrombosis, and stroke. He submitted an HNR on 11/12/18 (received on 11/13/18) 
stating that he fell over twice with his wheeled walker. The HNR should have been scanned into 
eOMIS by 11/15/18. Instead it was not scanned until 12/7/18 (along with another HNR which 
was received on 11/29/18 which itself should have been scanned by 12/1/18). The patient 
happened to be scheduled to see a provider on 11/27/18 for a related issue. Due to the delayed 
scanning of the 11/12/18 HNR (and confirmed by the lack of any mention of the two falls), the 
provider was unaware of the falls. In the absence of all the relevant information he needed about 
the patient’s condition, it was therefore impossible for the provider to fully and safely evaluate 
the patient’s condition on 11/27/18. 

In their comments on the draft of this report, Defendants note that the patient’s own behavior 
may have contributed to his condition and falls. 

PM 6 ( Provider orders will be noted daily with time, date, and name of person taking the orders 

off.) 

Patient 47 suffers from hepatitis C and a back injury in 2016 resulting in urinary incontinence at 
night. During a visit on 6/19/18, a provider believed the patient required an MRI to rule out 
serious spinal cord conditions such as pressure on the spinal cord or nerve roots. No one noted 
this order. The MRI was never ordered as an outcome of this visit. (An MRI was ordered 
1/17/19, but as the result of a de novo request.) If the patient had a serious spinal condition, a 
delay in diagnosis and treatment could have rendered the problem worse and irreversible. 

PM 10 ( Each patient's medical record will include an up-to-date Master Problem list.) 

Patient 33 had a visit with a provider on 12/27/18 in follow-up to an emergency room visit for a 
fall and pain and redness in his arm, for which he was diagnosed and treated for an infection 
(cellulitis). The provider failed to update the patient’s Master Problem List with this new 
diagnosis. Further, the provider affirmatively documented that she had in fact updated the 
problem list, when clearly she had not. It is critically important for every health care professional 
who cares for the patient to kn ow a patient’s history and conditions. Since is not always practical 
or possible to review every page of a patient’s medical record at every encounter, the problem 
list is a key element of a patient’s medical record because it provides, at a glance, an overview of 
the patient’s history and conditions. In the absence of an accurate list, an incorrect diagnosis or 
treatment plan may be implemented. For example, in the case of this patient, if the patient were 
to develop another infection in the weeks following his arm infection, selection of the proper 
(effective) antibiotic might be different depending on whether or not the patient had recently had 
another infection. 
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PM 11 (Newly prescribed provider-ordered formulary medications will be provided to the 
inmate within two business days after prescribed, or on the same day, if prescribed STAT '.) 
Patient 50 suffers from seizures for which he is prescribed levetiracetam. There were two lapses 
in compliance with PM 11. On 12/6/18 a provider ordered this medication to be continued by 
writing a new prescription. On 12/11/18 he received his last dose (from the old prescription). The 
medication (from the new prescription) does not appear to have been delivered to the facility. On 
12/31/18 the patient sent an HNR asking for his medication. On 1/3/19 a provider ordered it 
again, but he did not receive his first dose until three days later, on 1/6/19. This medication is 
prescribed to prevent the patient from having seizures. While often self-limited, seizures can 
result in injury, and rarely, death. The gap in medication - from 12/11/18 to 1/6/19 - placed the 
patient at significant risk of serious harm. The risk of a seizure was particularly high in the days 
following the de facto sudden withdrawal from the medication on 12/11/18. 

PM 15 (Inmates who refuse prescribed medication (or no show) will be counseled by a QHCP 
after three consecutive refusals.) 

Patient 48 suffers from diabetes, poorly functioning kidneys, bilateral leg amputation, heart 
disease, asthma, and hypertension He is supposed to receive clonidine twice daily if his blood 
pressure is elevated. On 12/4/18, 12/5/18, and 12/6/18 the patient refused the medication. (It is 
not clear if this was three or six consecutive refusals because despite the order to administer the 
medication twice daily, nurses only attempted to administer it once daily during this time span.) 
Failure to control blood pressure when it is elevated can have devastating consequences in such a 
patient, e.g. heart attack, worsening of kidney function. Counseling is intended to increase the 
likelihood that the patient will accept this (or an alternative) medication. Thus failure to counsel 
places the patient at significant risk. 

PM 23 (Automated External Defibrillators (AEDs) will be maintained and readily accessible to 

Health Care Staff.) 

and 

PM 24 (Emergency medical response bags are checked daily, inventoried monthly, and contain 
all required essential items.) 

PM 23 and 24 differ from most other PMs in that they are based on examination of equipment, 
not patient-related events. Thus I am unable to provide patient-specific examples of the dangers 
associated with non-compliance. However, based on my experience and knowledge: emergency 
response bags and AEDs are key pieces of equipment used for emergency medical care to help 
save lives; equipment in the bags can be missing or non-functioning and the AEDs can break or 
be missing; and routine checks helps ensure that equipment is not missing and functions when 
needed. Therefore failure to check AEDs and emergency response bags on a regular basis poses 
a significant risk of serious harm to a patient (or staff member or visitor) who has an emergency 
need. 
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PM 31 (. Mortality reviews will identify and refer deficiencies to appropriate managers and 
supervisors, including CQI committee, and corrective action will be taken.) 

Patient 46 died mid-year 2017 54 in his third decade of life from metastatic testicular cancer. 
About eight months earlier he had testicular pain for which an ultrasound yielded an abnormal 
result. Three weeks later a physician noted the test result and scheduled the patient to be seen a 
week later. That appointment was never scheduled. Two months after the ultrasound was 
performed, the patient submitted an HNR because of the lack of follow-up. The nurse who saw 
him referred his case to a provider to review. That referral never happened. Three months after 
the ultrasound, the patient submitted another HNR because of continued pain. The nurse who 
saw him referred him to a provider. A provider saw the patient two weeks later. The provider 
failed to review the patient’s medical record to discover the abnormal ultrasound (or to discover 
requests from the patient to be informed of the results of his test, which would have inevitably 
led to discovery of the ultrasound). Based on this information gap, the provider referred the 
patient to a general surgeon for repair of what he thought was a hernia. When seen by the 
surgeon - now four months after the ultrasound - the surgeon recommended an ultrasound of the 
scrotum (not having been informed by the referring provider that one was already done) and a 
referral to a urologist for what he deemed was a problem with the testicle (hydrocele - fluid on 
the testicle). The Corizon Utilization Management department, also failing to leam the patient’s 
full history, denied the referral because a hydrocele is a benign problem which can be treated by 
symptom management alone. The patient’s provider, still ignorant of the history, and ignoring 
the surgeon’s recommendation to perform an ultrasound, accepted the Utilization Management 
department’s denial and gave the patient a scrotal supporter. He instructed the patient to let staff 
know if he weren’t getting better. A month later, the patient made such a notification; his pain 
was now between 7 and 10 on a scale from 0 to 10. A nurse referred him to a provider who did 
not examine his scrotum, but referred him to a urologist. The referral was bereft of key 
information justifying the referral. The Utilization Management department again failed to obtain 
that information, again denied the request, and - now eight months after the ultrasound - the 
provider again accepted the denial. At nine months after the ultrasound, due to continuing pain 
and swelling, medical staff decided to obtain an ultrasound of the scrotum and abdomen. 
However, only the abdominal part of the ultrasound was ordered. Serendipitously, it showed an 
abnormality of the lungs (fluid around the lungs), which led to the diagnosis of metastatic cancer 
(of the testicle) in the lung. The patient died in the hospital shortly afterwards. Despite numerous 
system errors resulting in a young man dying from a potentially curable disease, and 
identification of many of those problems by the ADC Mortality Review Committee and 


54 I am omitting the exact date to ensure patient confidentiality because dates of death are 
searchable in the public record. 
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notification of them to Corizon for its June, 2018 55 mortality review meeting, Corizon failed to 
take any action at that meeting to correct any of the deficits. That these system errors pose a 
significant risk of serious harm is self-evident from the outcome in this patient’s case, and it 
follows that failure to address those problems in a timely manner 56 also poses the same risk to 
future patients. 

PM 35 (All inmate medications (KOP and DOT) will be transferred with and provided to the 
inmate or otherwise provided at the receiving prison without interruption.) 

Patient 42 suffers from HIV/AIDS, hepatitis C, schizophrenia, and asthma. He requires three 
continuous medications (two of them combined in one product; fluticasone/ salmeterol) and a 
fourth intermittent medication (“rescue inhaler”) to control his asthma. On 12/30/18 he was 
transferred to the Florence infirmary due to exacerbation of his asthma. The combination 
medication, which is supposed to be administered twice daily 57 , was not transferred with him. He 
did not receive his first dose until the morning of 1/3/19. Given the severity of his condition, 
absence of this medication for this long period of time increased the risk that his asthma 
exacerbation would not get better, or would deteriorate, which can result in death. 

PM 36 (A LPN or RN will screen HNRs within 24 hours of receipt.) 

The currently reported results for PM 36 show that performance is successful. However, in Part I 
of this report, I describe a significant flaw in the way PM 36 is measured (“receipt” is measured 
from the date the HNR is stamped, not the date it is likely received) and recommend that it be 
remeasured. Remeasurement of PM 36 may reveal unsuccessful performance, in which case it 
would need to have been addressed here in Part II. To that end, I am addressing it now; in the 
event that remeasurement of PM 36 reveals successful performance, this section of my report 
should be ignored. 

Patient 26 submitted an HNR dated 11/24/18 in which he wrote: “I need to talk to someone 
ASAP I’m locked in a cell alone all Day its Driving me crazy I feel like I’m going mad it gets 
harder to handle every day please help me.[sic]” The HNR was stamped in the medical unit on 
11/28/18. On 11/29/18 a nurse wrote on the HNR “Scheduled for Nursing Line - Refer to Mental 
Health.” There is no evidence the patient was seen by the nurse. The patient was, however, seen 
by a psychology associate on 11/30/18 who addressed the HNR. Thus there was an apparent 6- 


55 The long time span between the death in mid-year 2017 and notification to Corizon in June, 
2018 is neither ADC’s or Corizon’s fault. Based on PM 32 (A final independent clinical 
mortality review will be completed by the Health Services Contract Monitoring Bureau for all 
mortalities within 10 business days of receipt of the medical examiner’s findings.) the mortality 
review process relies on the medical examiner’s report, the timing of which is outside the control 
of ADC or Corizon. In Part IA of this report - and because of the long delays such as the one 
described in this example -1 recommend that the mortality review process proceed with or 
without the medical examiner’s report. 

56 I did not review records subsequent to June 2018 to learn if Corizon took action at a later time. 

57 The patient is ordinarily allowed to keep this medication on his person (“KOP”), however, 
upon admission to the infirmary it was converted to a nurse-administered medication. 
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day delay from when the patient submitted the HNR - according to the date he wrote it - until he 
was seen, four days of which are accounted for by the gap between the date he wrote on the HNR 
and the date it was stamped by medical staff. On 12/5/18, he submitted another HNR in which he 
wrote, “This is the second kite I’ve put in I need to talk to someone ASAP I’ve become very sad 
this last week I’m all alone & cant stand it much longer please help me.[sic]” He was finally seen 
by a nurse on 12/8/18. The patient’s symptoms were significant and indicated the possibility of 
self-harm. Thus the delay in addressing his original request placed him at significant risk of 
serious harm. 

PM 37 (Sick call inmates will be seen by an RN within 24 hours after an HNR is received (or 
immediately if identified with an emergent need, or on the same day if identified as having an 
urgent need). 

Patient 23 presented to the clinic on 12/2/18 around noon because he was hit in the eye that 
morning. He was evaluated by an LPN. The LPN reported that the patient denied any change in 
vision, nausea, vomiting, or dizziness, and “denie[d] any other injuries or other medical 
concerns.” The only examination the LPN conducted was to note a one half inch red mark on his 
left eyebrow. The LPN thought it was most appropriate to examine the patient under a 
“Musculoskeletal” nursing guideline, rather than a guideline focusing on head or eye injury. 
Regardless, having chosen the guideline, the LPN then ignored the history part of the guideline, 
except for noting the level of pain and the fact that nothing made the pain better or worse; the 
LPN also ignored the physical examination part of the guideline entirely. She also failed to 
obtain vital signs. Not only was the nurse’s evaluation wholly insufficient, it was conducted 
independently, without input from an RN or provider. While the LPN wrote that she would defer 
the filling out of the Assessment Notes to an RN, she made no referral to an RN, and no RN ever 
saw the patient after the LPN discharged the patient back to his living unit with no plan other 
than providing some unspecified “pt education].” Though the patient denied any change in 
vision, he may have suffered damage to his eye that would only be evident on examination (e.g. 
an objective check of his vision, looking in the eye for signs of bleeding). By definition, this 
patient also suffered a head injury. Although a patient might report feeling alright, he or she may 
have suffered brain damage requiring immediate attention. Thus additional examination and 
possible monitoring and diagnostic testing may have been required. Thus independent (in 
violation of the nurse’s scope of license and training) - and incompetent - management of this 
acute injury by an LPN posed a significant risk of serious harm. 

PM 39 ( Routine provider referrals will be addressed by a Mediccd Provider and referrals 
requiring a scheduled provider appointment will be seen within 14 calendar days of the 
referral.) 

Patient 36 submitted an HNR on 12/27/18 due to increasing pain. A nurse saw him on 12/31/18. 
She found that he had a hernia the size of a “grapefruit” which was tender to touch on 
examination. She referred the patient to a provider. The patient was not seen by the provider until 
1/17/19. The provider discovered the hernia was incarcerated (not able to be pushed back into the 
abdomen) which is a danger sign for hernias. Therefore she referred him to a surgeon. The 
urgency of the need for surgery is reflected by the surgeon’s conclusion when he saw the patient 
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on 2/27/19: “Patient needs incarcerated right inguinal hernia repair with prolene mesh ASAP!” 
Incarceration of a hernia is a serious complication of hernias because the intestines - the contents 
of the hernia bulge - can become cut off from its blood supply, which is a life-threatening 
emergency. The delay in having the patient seen by the provider and ultimately referred to the 
surgeon therefore placed the patient at significant risk of serious harm. 

PM 40 ( Urgent provider referrals are seen by a Medical Provider within 24 hours of the 
referral.) 

Patient 37 suffered an acute injury of his leg on 1/13/19. He saw a nurse for the second time on 
1/21/19 due to increasing pain and bruising. The nurse found an 8 x 6 centimeter bruise on his 
inner thigh and pain between 2 to 7 on a scale from 1 to 10. The nurse consulted with a provider, 
and due to concern for a serious condition (e.g. fracture, severe bleeding), the provider ordered 
an immediate x-ray and for the patient to return to see a provider within 24 hours. Instead, the 
patient was not seen by a provider until two days later. If the patient had indeed suffered from a 
serious condition as a result of the injury, the delay in seeing a provider posed significant risk of 
serious harm, such as severe bleeding, pain, soft tissue damage. 

PM 42 (A follow-up sick call encounter will occur within the timeframe specified by the Medical 
or Mental Health Provider.) 

Patient 17 was seen by a provider on 12/1/18 for pain in his groin and burning on urination. The 
provider’s working diagnosis was that the patient had an infection, and ordered appropriate 
infection testing and treatment. She ordered for the patient to have a follow-up appointment with 
a nurse in six weeks (by 1/12/19) to check on his condition. No such appointment ever took 
place. On or around 1/23/19, the patient submitted an HNR complaining that his pain was getting 
worse and was seen on 1/24/19 by a nurse in response to this HNR. The nurse concluded that he 
needed to see the provider, which took place on 2/2/19. In light of his lack of response to 
treatment and negative tests for infection (which were available by 12/4/18), the provider 
referred the patient to a urologist due to concern for a possible stricture of the urinary tract. A 
stricture is a serious condition which not only causes pain, but can increase the chances of 
urinary tract infection and possible loss of kidney function. Thus the failure to execute the 
follow-up encounter with the nurse by 1/12/19 delayed the patient’s care and posed a significant 
risk of serious complications if he had a stricture. 

PM 43 (Inmates returning from an inpatient hospital stay or ER transport will be returned to the 
medical unit and be assessed by a RN or LPN on duty there.) 

Patient 9 was re-admitted to ADC on 6/21/18 after having been hit in the head with a brick 
somewhere outside ADC. Later that day, he was sent from ADC to the emergency room where it 
was determined that he had a concussion, laceration, and fractured nose. He was sent back to 
ADC that evening with instructions to be observed for the next 24 hours for any mental status 
changes and to take medications, as needed, for pain. However, he was not seen by any health 
care staff upon arrival at ADC until the next day (approximately 12 hours later). Thus, at least 
for the first half of the 24 hour concussion observation period recommended by the hospital, he 
was without observation and without pain medications. During this unobserved period he was at 
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risk for complications of the concussion such as loss of consciousness, brain damage, and death. 
Thus the failure to assess him upon return from the hospital placed him a significant risk of 
serious harm. 

PM 44 ( Inmates returning from an inpatient hospital stay or ER transport with discharge 
recommendations from the hospital shall have the hospital’s treatment recommendations 
reviewed and acted upon by a medical provider within 24 hours.) 

Patient 18 suffers from diabetes, hypertension, asthma, and heart disease. On 12/6/18 he returned 
from a hospital admission where he was sent for chest pain and elevated blood pressure. The 
hospital discharge instructions, among others, recommended the patient be placed on 
nitroglycerin as needed for chest pain and metoprolol 25 mg twice daily (for heart and blood 
pressure). The recommendation for nitroglycerin was ignored (the patient was eventually given a 
prescription on 1/24/19) 58 . The recommendation for metoprolol 25 mg twice daily was partially 
ignored; instead the patient was prescribed the medication only once a day. Further, it does not 
appeal - he received the medication until at least 12/10/18. Both medications play an important 
role in protecting the heart of a patient such as this one from having a heart attack. Failure to 
execute the recommendations of the hospital placed the patient at significant risk of serious 
harm. In fact, four days after returning from the hospital, the patient’s blood pressure rose 
dangerously and he experienced another emergency with chest pain. 


PM 45 ( On-site diagnostic services will be provided the same day if ordered STAT or urgent, or 
within 14 calendar days if routine.) 

Patient 3 suffers from anemia, hepatitis C, morbid obesity, diabetes, and hypertension. Due to 
intestinal problems, he receives all nutrition via intravenous tubes (TPN; total parenteral 
nutrition) in the infirmary. Because the levels of critical minerals in the blood can become 
abnormal when receiving TPN, the levels must be monitored. One such mineral is magnesium. 
An abnormally low level of magnesium can cause seizures and death. A provider ordered a 
STAT magnesium level to be performed on 12/10/18. The order was never executed; the 
magnesium level was never checked. This places the patient at significant risk of serious harm. 

PM 46 (A Medical Provider will review the diagnostic report, including pathology reports, and 
act upon reports with abnormal values within five calendar days of receiving the report at the 
prison.) 

Patient 57 was seen by a nurse on 12/8/18 for a head injury with laceration following a fight. A 
provider instructed the nurse to obtain x-rays of the face, neck, and skull. The x-ray results were 
reported back to the facility on 12/11/18, but the provider did not review the results until 
12/18/18. The reason for obtaining the x-rays was to be sure the patient did not have a fractured 
skull, spine, or face, any of which would have constituted a medical emergency. In retrospect the 


58 The patient was last given nitroglycerin in May, 2018. While it is conceivable that he still had 
this medication in his possession, by December, the medication would have lost its potency. 
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patient did not have any fractures; however, this was not known by the provider until 12/18/18, 
and thus the delay in reviewing the results placed the patient at significant risk of serious harm. 

PM 48 ( Documentation, including the reason(s) for the denial, of Utilization Management 
denials of requests for specialty services will be sent to the requesting Provider in writing within 
fourteen calendar days, and placed in the patient's medical record.) 

Patient 5 was seen by a provider on 10/29/18 for right testicular pain and right abdominal pain. 
On examination he had tenderness in his abdomen and a hard nodule on his right testicle. The 
provider requested an ultrasound of the abdomen and testicle. On 12/9/18 - six weeks later, and 
four weeks after the required deadline - the request was denied by Corizon’s Utilization 
Management department. With regard to his testicular nodule, the most important diagnosis 
which needed to be ruled out was a testicular cancer, and the ultrasound was the correct test to 
order. The delay in denying this request delayed any appeal that the requesting provider would 
have (rightfully) made, delaying subsequent treatment of a cancer, which would have 
significantly increased the risk of treatment failure of cancer. 59 

PM 49 ( Patients for whom a provider’s request for specialty sendees is denied 60 are told of the 
denial by a Medical Provider at the patient’s next scheduled appointment, no more than 30 days 
after the denial, and the Provider documents in the patient’s medical record the Provider’s 
follow-up to the denial.) 

This PM measures the degree to which providers keep patients apprised of changes in their 
treatment plans. As reflected in the Noncompliant performance levels on this PM at a number of 
ADC complexes, it is clear that some medical staff are not completing this task appropriately. It 
is good medical practice to keep patients apprised of changes in their treatment plan. However, 
the question posed by the Court is whether failure to do so poses a significant risk of serious 
harm. Based on my experience with, and knowledge of, the UM process, and my review of many 
denied provider requests at ADC, I am unable to conclude that failure to notify the patient of 
denials within 30 days rises to that level. The most compelling risk potentially occurs in the 
following hypothetical scenario: 

The Alternative Treatment Plan includes a trip to an outside specialist or diagnostic 
service (e.g. colonoscopy) different from the one the patient was expecting based on the 
original plan (e.g. a plain x-ray of the abdomen). No one has informed the patient of the 
change in plan. On the day of the trip to the specialist for the colonoscopy, the patient 
refuses the trip because that was not what he was expecting, and also because the test is 
something more involved or aggressive than what the patient was expecting. The 
colonoscopy is cancelled for that day. 

59 It should be noted that the requesting provider - inappropriately - did not appeal the decision. 
However, this does not change my analysis which is based on what a reasonable provider would 
have done. 

60 As will be explained in more detail later in this report, most denials of provider requests for 
specialty services are not labelled explicitly as “Denials.” Instead, the decision-maker - part of 
the Utilization Management department - usually recommends an Alternative Treatment Plan. 
Thus it is a de facto denial of the original request. 
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However, I opine that failure to notify the patient within 30 days does not pose a significant risk 
of serious harm for the following reasons: (a) of the many ATPs I reviewed, I found none where 
the plan was for a specialty service which was more aggressive than the one the patient was 
expecting, so a patient refusal is less likely; (b) the scenario above, while possible, is rare; and 
(c) if the patient did refuse, and if the refusal is handled the way it should be, a provider would 
immediately be notified who would speak with the patient and hopefully encourage the patient to 
accept the specialty service. 

In their response to the draft of this report, Plaintiffs note that as I “noted [in PM 48], the delay in 
telling the provider of the denial delays their ability to take action.” This is accurate, but does not 
load directly on PM 49. The Plaintiffs also note that “The failure to document the provider’s 
follow-up actions to the denial also can pose a significant risk of harm because it can lead to 
delays and also repeated requests for specialty care.” This comment pivots on the last phrase of 
the PM - “ and the Provider documents in the patient’s medical record the Provider’s follow-up 
to the denial ” and assumes that the follow-up referred to is the follow-up care the provider orders 
for the patient instead of the original care request. However, I do not believe that is how this PM 
was intended or is used. The Monitoring Guide (revision 2/7/18) to which the Parties agreed, 
states that the “CGAR Question” for monitors to address when auditing this PM is simply, “Are 
patients for whom a provider’s request for specialty services is denied told of the denial by a 
Medical Provider at the patient’s next scheduled appointment, within thirty (30) days of the 
denial?” In the Methodology section of the Guide, the sole determination the monitor is 
instructed to make is: “To be considered compliant, compare the source document UM ATP 
Date with the next patient/provider documented appointment. If that appointment occurred 
within thirty (30) days, that consult shall be identified as compliant.” Finally, I examined what 
the effect would have been on PM 49 if “Provider’s follow-up to the denial” had been interpreted 
as meaning the care the provider planned to provide in place of his or her original specialty 
request. I reviewed a random sample of 11 cases which were found Noncompliant during the 
December, 2018 CGAR. In all 11 cases, the provider had a follow-up plan of care and 
documented it (in the form of acceptance of the Alternative Treatment Plan proposed by the 
Utilization Review Committee). In summary, while failing to design and document a follow-up 
treatment plan would pose a significant risk of serious harm, that is not what PM 49 measures, 
and therefore my opinion above is unchanged. A related issue - the inadequacy of the 
Alternative Treatment Plans that the providers do accept - is discussed in Part IV of this report in 
the section “ Utilization Management (UM) Process - Part 2: Managing Patients after Denials of 
Specialty Referral Requests.” 

PM 50 ( Urgent specialty consultations and urgent specialty diagnostic sendees will be scheduled 
and completed within 30 calendar days of the consultation being requested by the provider.) 
Patient 30 has congestive heart failure requiring an implanted pacemaker/defibrillator in his heart 
to keep his heart beating at the proper speed. During an encounter on 11/19/18, a provider noted 
worsening heart failure and a mechanical problem with the pacemaker (bent wire). She submitted 
an “Urgent” consultation to the cardiologist. This consult should have been completed by 
12/19/18; instead it was not completed until 1/18/19, a month late. The role of a pacemaker is to 
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keep the heart beating, or restart it, if it stops. The heart is more likely to stop when heart failure 
is present. Thus the longer the patient went without repair of the pacemaker, the greater the risk 
of serious harm. 

PM 51 ( Routine specialty consultations will be scheduled and completed within 60 calendar days 
of the consultation being requested by the provider.) 

Patient 16 has increased pressure in his eyes, a condition which is the main risk factor for 
glaucoma. On 10/11/18, a provider referred him to an ophthalmologist for an interval recheck of 
the pressure level. The Utilization Management department of Corizon recommended an 
alternate treatment plan: that the patient should see an optometrist instead of an ophthalmologist 
(a reasonable alternative). On 10/13/18 the provider accepted this plan and generated a new 
request for the patient to see an optometrist. The consult should have been completed in 60 days 
(i.e., by 12/11/18). On 4/10/19 it was cancelled. The reason given was that the patient was 
transferred to another facility (it appears that on 2/18/19 he was transferred from the facility 
where the consult was generated (Yuma) to Eyman). As of the date of transfer, the consultation 
was already more than a month overdue. Further, as Corizon is the health care vendor at both 
Yuma and Eyman, and the patient’s medical record is electronic and continues to exist, 
seamlessly, across interfacility transfers, “transfer” is not a valid reason for non-performance. As 
of late May, 2019, no consultation has been performed nor new request generated. Increased eye 
pressure is the major risk factor for glaucoma. Glaucoma is a serious condition which, if left 
untreated, can cause blindness. Early intervention, achieved by monitoring patients with 
increased pressure and providing medications or surgery when necessary, can prevent most 
patients from losing their eyesight. Thus the failure to conduct this consultation placed the 
patient at significant risk of serious harm. 

PM 52 ( Specialty consultation reports will be reviewed and acted on by a Provider within seven 
calendar days of receiving the report.) 

Patient 1 was the recipient of a kidney transplant. He saw a kidney specialist, the report of which 
was received on 10/22/18. PM 52 required review of the report by 10/29/18. The facility 
physician did not review the specialist’s report until 11/8/18. The care of a patient with a 
transplanted kidney is quite complex, and delays in recommendations made by the specialist put 
the patient at significant risk of kidney failure and loss of the kidney. 

PM 53 ( Treatment plans will be developed and documented in the medical record by a provider 
within 30 calendar days of identification that the inmate has a chronic disease.) 

Patient 11 was found to have hypertension (blood pressure 161/99) upon transfer on 11/15/18. A 
nurse contacted a provider who ordered the patient started on a medication for hypertension, and 
for the patient to have a follow-up blood pressure check with the nurse on 11/22/18. This latter 
appointment never transpired. The patient received no other care for or monitoring of his blood 
pressure, or any treatment plan, until he had his first chronic care visit on 1/7/19. Given the 
abnormally elevated blood pressure and the fact that it had not been monitored in the previous 
eight months (i.e., did not have a predictable track record), in the absence of a treatment plan 
including patient education and monitoring, the patient’s condition went with insufficient care 
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for more than 50 calendar days. During this period of time, he was at significant risk for his 
blood pressure rising to very high levels which would could cause acute heart, brain, and kidney 
damage. 

PM 55 ( Disease management guidelines will be implemented for chronic diseases .) 61 
Patient 6 was admitted to ADC on 9/6/18 with a history of diabetes, hepatitis C and 
hypertension. Providers did not attempt to obtain blood tests to assess the status of his diabetes 
(HbAlc) or hepatitis until 1/16/19 and did not obtain a diabetic eye exam until 3/4/19. The 
chronic care guidelines for diabetes required the patient to have a diabetes blood test every three 
months. There was no record of the patient having had any such test prior to arrival, thus the test 
was due upon arrival on 9/6/18; when performed on 1/16/19 it was more than four months late. 
This test is necessary to assure that the patient’s diabetes is under control, and if it is not, to 
guide changes of therapy. Uncontrolled diabetes can cause short and long term complications, 
including damage to organs and limbs, and death. The chronic care guidelines for diabetes also 
required the patient to have an annual eye exam. There was no record of the patient having had 
such an exam prior to arrival, thus the exam was due upon arrival on 9/6/18; when performed on 
3/4/19 it was six months late. Blindness is a known complication of diabetes which is largely 
treatable if found early; thus periodic exams are essential to reducing blindness. The chronic care 
guidelines for hepatitis C required the patient to have lab tests every six months. There was no 
record of the patient having had any such tests prior to arrival, thus the tests were due upon 
arrival on 9/6/18; when performed 1/24/19 they were more than four months late. These tests are 
necessary to assess damage caused by the hepatitis and guide whether therapy is necessary. 
Untreated hepatitis C can cause damage to the liver including cirrhosis and cancer. Delays in all 
three types of care described above increased the chances of the complications described and 
thus placed the patient at significant risk of serious harm. 


PM 57 ( A Medical Provider will order prenatal vitamins and diet for a pregnant inmate at the 
inmate's initial intake physical examination.) 

I reviewed all (four: Patient 4, Patient 10, Patient 25, Patient 31,) Noncompliant cases going back 
to May, 2018. In two of the cases, I actually found them to be Compliant. In the third case, the 
prenatal vitamins were ordered a week late, but the patient was already well out of the critical 
period when vitamins (specifically folic acid) are needed. The diet was ordered four weeks late, 
but the patient was still close to the beginning of the period when a special diet is recommended, 
and further, the evidence that any change in diet is needed for someone receiving an otherwise 
nutritious diet is weak. In the fourth case the diet was ordered a week late. Thus these cases of 
noncompliance posed either no risk or minimal risk. 


61 PM 54 (Chronic disease inmates will be seen by the provider as specified in the inmate's 
treatment plan, no less than every 180 days unless the provider documents a reason why a longer 
time frame can be in place.) is very similar to PM 55, and thus I discuss only PM 55 here. I 
discuss the overlap between these two PMs in Part IA. 
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PM 59 ( Inmates will be screened for TB on an annual basis.) 

Patient 24 had his last annual TB screening on 8/5/15. It was negative. He should have received 
his next annual screening on or about 8/5/16. The test was administered on 7/22/16, but the result 
(whereby a nurse examines the patient’s skin where the injection was administered and 
documents a negative or positive reaction) was never obtained. As of late May, 2019, more than 
3.5 years after it should have been completed according to PM 59, the screening test for TB has 
not been repeated. 

The screening test required by PM 59 is a skin or blood test. It will identify people with active 
TB (as long as they have had the infection long enough), but symptom-based screening (not 
covered by this PM) is the more useful tool for identifying active TB. The main role of the 
screening test in PM 59 is to identify people with dormant (latent) TB. Annual screening for TB 
is done partially for the safety of the tested individual: if the test is positive it means, most likely 
the patient has dormant TB, which has a 10% chance of reactivating to active TB over the 
person’s lifetime. So they are offered treatment. The screening is also done for the safety of the 
prison’s other residents and staff to help monitor for the introduction of TB into the prison. TB is 
a serious disease with dangers both from the disease as well as the treatment. 

However, the question remains whether failing this PM poses a significant risk. This risk is 
dependent on the underlying likelihood of TB occurring in the prison in the first place. For 
prisons with low likelihood (deemed “minimal risk,” based on an assessment outlined by the 
CDC), annual testing may not be needed. The risk would also be informed by the historical data: 
the frequency with which active and dormant TB cases where identified in the past at the 
noncompliant facilities. I did not collect enough data to make the assessment whether ADC, or 
more specifically, the complexes which have been Noncompliant with PM 59, meet the CDC 
definition of minimal risk. It is also important to note that on 5/17/19, CDC revised its 
recommendation for annual TB screening among health care personnel, stating that it was no 
longer necessary “[i]n the absence of known exposure or evidence of ongoing TB 
transmission...” (MMWR 68(19);439-443) While not wholly on point with the question at hand, 
it is informative. Thus I am unable to state with certainty whether noncompliance with PM 59 
poses a significant risk of harm. 


PM 63 (In an IPC [Inpatient Component; Infirmary], an initial health assessment will be 
completed by a Registered Nurse on the date of admission.) 

Patient 51 was admitted to the IPC at approximately 14:00 on 6/29/18 upon return from the 
hospital for a change in his level of consciousness, low blood pressure, and an infection in his 
lung and blood. The initial nursing health assessment was not conducted until after midnight on 
6/30/18. At the end of the assessment, the nurse concluded that the patient had a moderate risk of 
falling. The nursing care plan thus needed to include steps to prevent falls. Falls can have serious 
consequences in any patient, but especially in someone like this patient who was 66 years old 
and quite compromised physiologically. Beyond being conducted on the next day after 
admission, more importantly, the assessment was not conducted until more than 10 hours later. 
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During these 10 hours - before the nurse could implement her care plan to reduce the risk of 
falling (e.g. she put his bed in a low position and made sure the call light was within reach) - the 
patient was at increased risk of serious harm. 

PM 64 {In an IPC, a Medical Provider evaluation and plan will occur within the next business 

day after admission.) 

and 

PM 65 {In an IPC, a written history and physical examination will be completed by a medical 
provider within 72 hours of admission.) 

PM 64 and 65 measure the same clinical process, so I provide a single example for both here. I 
discuss their overlap in Part IA. 

Patient 52 was admitted to the IPC on 12/11/18 (a Tuesday) because his medical needs could not 
be met at this previous facility. He has a history of hepatitis C, hypertension, abnormal thyroid 
function, and schizophrenia. At the time of admission he was combative and incontinent. The 
medical provider conducted his evaluation and crafted his care plan on 12/17/18. It was on this 
date that he first instructed nurses on how to monitor the patient’s condition. While the patient 
was well-known to the staff at the IPC, his condition had changed (driving the transfer to the 
IPC), and thus he required provider evaluation both to look for any serious problem which may 
have led to the change, as well as to instruct nurses on a plan of care. During the week the patient 
resided in the IPC without the benefit of a provider evaluation and plan, he was at risk for 
complications from myriad serious medical diseases which may have led to his change or which 
may have occurred in the interim. 

PM 66 {In an IPC, a Medical Provider encounters will occur at a minimum every 72 hours.) 
Patient 55 was admitted to the IPC on 12/2/18 following repair of a fractured pelvis and thigh. 

He also suffers from seizures, hypertension, asthma, depression, and schizoaffective disorder. A 
provider attempted to perform an initial evaluation on the morning of 12/3/18, but was unable to 
complete it because the patient became verbally abusive and refused the evaluation. There is no 
evidence that the provider assessed the patient’s mental status, his capacity to make medical 
decisions in his own best interest, or attempted to conduct an informed refusal. Given the fact 
that the current behavior represented a change in behavior from the previous day (when the 
patient was cooperative with the admitting nurse) and that he had just had major trauma followed 
by major surgery, it was incumbent on the provider to consider whether the patient may have 
been suffering from any serious complication of the trauma or surgery which can result in 
changes in mental status. There is no evidence this was done. No provider attempted to evaluate 
the patient again until the afternoon of 12/7/18. The delay of five days in seeing the patient 
placed him at significant risk of serious harm from an untreated complication of trauma or 
surgery. 
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PM 67 {In an IPC, Registered nurses will conduct and document an assessment at least once 
every shift. Graveyard shift assessments can be welfare checks.) 

Patient 20 was admitted to the IPC on 12/28/18 after having been released from the hospital for 
heart failure and low blood pressure. According to the nursing care plan, he was at risk for going 
back into heart failure and required monitoring of his fluids and lungs. He had an assessment 
conducted on 12/30/18 at 14:54 during day shift. The night shift nurses failed to conduct an 
assessment (or welfare check). His next nursing assessment was not until 12/31/18 at 14:16, 
almost 24 hours later. The heart can go into failure rather quickly and rapid recognition of the 
problem and treatment can greatly reduce the likelihood of decompensation and death. Therefore 
failure to assess this patient on a more frequent basis, as foreseen by the PM, placed him at 
significant risk of serious harm. 

PM 68 {In an IPC, Inmate health records will include admission orders and documentation of 
care and treatment given.) 

Patient 45 was admitted to the IPC on 2/18/18 at approximately 19:00 following discharge from 
the hospital after surgery for fractured bones of the eye socket. The discharging surgeon 
recommended that the patient be continued on a number of medications, including oxycodone for 
post-operative pain, a pureed diet, antibiotic mouth wash prior to each meal and bedtime, 
antibiotic pills, and saline rinsing of the nose. The pain medication was not ordered because it 
was documented that the facility does not provide it. No attempt was made to obtain it or find an 
equivalent substitute (the following day the patient was offered a different medication, which is 
half as strong as what the surgeons recommended); the patient did not receive any pain 
medication until the following day, almost a full day after leaving the hospital and receiving his 
last dose of pain medication. The mouthwash, given to prevent oral bacteria from passing 
through the surgical wound and causing a deep infection in the patient’s head, was not provided 
as prescribed before meals: the “pre-lunch” dose was prescribed and given at 2 PM. Finally, 
none of the other medications - including the post-operative antibiotic, also meant to prevent an 
infection within the head due to surgery, which was likely due the evening of 2/18/18 - were 
provided until the next morning. Delayed timing of antibiotics decreases their effectiveness. 
These failures to provide the recommended treatment posed a significant risk of serious harm - 
pain and infection - to the patient. 

PM 77 {Mental health treatment plans shall be updated a minimum of every 90 days for MH-3A, 
MH-4, and MH-5 prisoners, and a minimum of every 12 months for all other MH-3 prisoners.) 

I reviewed, with Dr. Abplanalp’s assistance, Noncompliant cases at Tucson audited for the 
CGARs of September 2017 (the most recent CGAR for which any facility was Noncompliant). 
Of the 14 cases I reviewed, one had no treatment plan updated and 13 had updates, but they were 
late. The case with no update was a patient classified as MH-3B with depression and anxiety 
who released to the community when the update was four months overdue. For the 13 other 
patients, late updates ranged between (least severe) two days late for an annual update to (most 
severe) two months late for a quarterly update. Most late updates clustered near the less severe 
end of the range. 


86 





Case 2:12-cv-00601-ROS Document 3379 Filed 10/04/19 Page 87 of 138 


I was unable to identify a significant risk of serious harm due to noncompliance in any of the 
cases. This is due, in part, to the fact that, with perhaps the exception of the one non-updated 
case, the delays were not clinically significant, and treatment continued, despite the absence of 
an updated plan. My review did, however, reveal a marked lack of comprehensiveness of many 
of the extant treatment plans. However, as this is not an issue measured by PM 77,1 address it in 
Part IV of my report. 

It should be noted that all complexes, except Tucson and Phoenix, have been compliant with PM 
77 for over 3.5 years. Phoenix was Noncompliant once in December, 2016. Tucson, with the 
worst record, had four Noncompliant months in a row from June to September, 2017. However, 
as of March, 2019, it has been Compliant for the past 18 months. 

PM 94 {All prisoners on a suicide or mental health watch shall be seen daily by a licensed 
mental health clinician or, on weekends or holidays, by a registered nurse.) 

Patient 15 has a history of bipolar disorder and generalized anxiety disorder. On 10/31/18 a 
licensed mental health clinician placed him on 10-minute mental health watch because he was 
“decompensating” psychologically (“Client appears to be functioning below his psychological 
baseline. There was overt evidence of significant disturbance in thought, mood, and behavior. He 
presents as a danger to self at this time due to possible retaliation by other inmates due to clients 
erratic behavior and inappropriate speech.”). He was seen on 11/1/18 and 11/2/18 by an 
unlicensed mental health clinician who, on the latter date, determined it was safe to advance the 
patient to 30-minute watches. On 11/5/18 and 11/6/18 he was seen again by an unlicensed mental 
health clinician. Neither the 11/2/18 visit resulting in a promotion from 10-minute to 30-minute 
watch, nor any of the other three visits were conducted in a confidential setting (the patient was 
offered and refused). There is no evidence that any of the four visits conducted by an unlicensed 
clinician were contemporaneously or post hoc approved, or even reviewed, by a licensed 
clinician. Advancement from one levels of watch to the next level is a critical decision requiring 
careful assessment of the patient’s mental status harm because an error in this decision to 
observe the patient three fold less frequently (30 minutes vs. 10 minutes) could result in the 
patient finding an opportunity to harm him/herself. Thus performance of this task by a clinician 
who is unlicensed to operate independently in the prison, without any supervision or 
collaboration from a licensed clinician, poses a significant risk of serious harm. 

PM 95 {Only licensed mental health staff may remove a prisoner from a suicide or mental health 
watch. Any prisoner discontinued from a suicide or mental health watch shall be seen by a 
mental health provider, mental health clinician, or psychiatric registered nurse between 24 and 
72 hours after discontinuation, between seven and ten days after discontinuation, and between 
21 and 24 days after discontinuation of the watch.) 

Patient 38 has a history of major depressive disorder. He was placed on watch after stating he 
was hearing voices "telling me to kill myself, to hang myself." He reported a history of 
intermittent auditory hallucinations "for a couple years, but they come in waves; this morning it 
got really intense,” and also reported that the voices instruct him to hurt "people that are threats, 
like other inmates.” His watch was discontinued by an unlicensed mental health staff member 
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without supervision on 10/15/18. The decision to discontinue watch is a critically important one, 
and its completion by a person not yet qualified to do so put the patient at significant risk of self- 
harm. 
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Part III - Causes of substantial noncompliance, the barriers to compliance, and 
recommendations to alleviate them. (Doc. 3089 at 2) 

In its charge, the Court asked me to address causes of noncompliance, including but limited to 
the following areas of health care delivery: 

• Pharmacy (PM 15 at Lewis; PM 19 at Lewis); 

• Intersystem Transfers (PM 35 at Lewis); 

• Access to Care (PM 39 at Lewis; PM 40 at Eyman; PM 42 at Eyman, Florence, Lewis; 
PM 44 at Eyman, Florence, Lewis); 

• Diagnostic Services (PM 46 at Eyman; PM 47 at Eyman, Lewis, Phoenix, Tucson); 

• Specialty Care (PM 49 at Tucson; PM 50 at Florence, Tucson; PM 51 at Florence; PM 52 
at Eyman, Florence, Tucson); 

• Chronic Care (PM 54 at Eyman; PM 55 at Eyman); and 

• Infirmary Care (PM 66 at Florence; PM 67 at Lewis, Tucson). 

Elsewhere in my report, primarily Part I and Part IV, I describe other areas of health care 
delivery where compliance is (or might be, after recalculation) poor. Addressing these 
deficiencies would ordinarily elicit problem-specific recommendations. However, with a few 
exceptions (see below), I have concluded that the deficiencies discussed in my report are best 
addressed with some overarching recommendations which touch almost all the deficiencies. 

A key realization that led me to this approach is that, with the few exceptions, these deficiencies 
do not exist because of a knowledge deficit on the part of Corizon 62 that requires the expertise of 
an external expert to fix. Corizon is a mature company which has been in the correctional health 
care business (as Corizon, or as its progenitors Prison Health Services and Correctional Medical 
Services) for years and has faced challenges greater than the ones it faces in Arizona. Corizon 
has the experience and expertise it needs. In my opinion, failure to fix the problems identified by 
the Court and identified in this report stems from a lack of will, rather than a lack of know-how 
on the part of the health care vendor. So - with the exception of the PMs noted in the next 
paragraph - I believe it will be more useful for me to concentrate on the macro-level barriers and 
solutions. 

Of the areas of health care delivery cited by the Court (listed above), the following merit specific 
comments: 

• Pharmacy, PM 19 at Lewis: In Part IA of my report, I recommend retiring this PM. 

• Intersystem Transfers, PM 35 at Lewis: In addition to the overarching barriers below, the 
way PM 35 is calculated makes it more difficult to identify the sending facility 


62 In July, 2019, late in the drafting of this report, the vendor changed from Corizon to Centurion. 
Almost none of the data upon which I based my report is drawn from work performed by 
Centurion. However, as Centurion carried over most staff and inherited most of the structural and 
functional elements of the previous contract, I believe my opinions remain relevant. 
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responsible for mishandled transfers and therefore more difficult to fix the problem at its 
root. This is discussed in Part IA of my report. 

• Access to Care, PM 44 at Eyman, Florence, Lewis: In Part I of my report I explain that 
these facilities’ level of performance may be over- or under-stated; in the latter case the 
issue of barriers would be moot. 

• Diagnostic Services, PM 46 at Eyman: In Part I of my report I explain that these 
facilities’ level of performance may be over- or under-; in the latter case the issue of 
barriers would be moot. 

• Specialty Care, PM 50 at Florence, Tucson and PM 51 at Florence: In addition to the 
overarching barriers below, a cumbersome UM process and reduced willingness of 
community specialists to see prison patients are specific barriers to performance, as 
explained below in the sections entitled Redesign the Process for Fulfilling Provider 
Requests for Specialty Services and Increase Community Specialist Fee Payments, 
respectively. 

I present the macro-level barriers and solutions below. By far the most critical barrier to ADC’s 
compliance with the PMs in this case is insufficient funding of health care services. After 
describing that deficiency, I describe several areas where increased funding should be applied. 
Next I describe the second most critical barrier to ADC’s compliance: its privatization of health 
care services. Finally, I describe several other important barriers to compliance with the 
PMs and recommendations to alleviate them. 

Insufficient Funding 

Increase Per Capita Health Care Expenditure 

Issue: 

While money by itself cannot fix everything, it is impossible to provide safe health care 
in the absence of adequate spending. I thus endeavored to determine if ADC’s current 
spending level for health care is adequate or whether insufficient spending is a barrier to 
compliance. 

One approximation of the adequacy of ADC spending can be gleaned from a comparison 
of Arizona to other state prison systems. For this comparison I used data from a recent 
Pew report 63 . It shows that ADC spent the 6 th lowest amount for health care, per capita, 
compared to the 49 other states reporting: ADC spent $3,529 per year per resident; the 
median was $5,720 (see figure below). While the absolute amount is no longer valid in 
2019 due to inflation, the relative amount probably is. 


63 Prison Health Care: Costs and Quality. Pew Charitable Trusts. October, 2017, available at 
https://www.pewtrusts.org/en/research-and-analysis/reports/2017/10/prison-health-care-costs- 
and-quality. In the interest of full disclosure, I served as an external reviewer of that report. 
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However, this approximation is hampered by some limitations. First, it compares Arizona 
against other states which may have different costs of living and different disease 
profiles. Second, it is likely that when reporting to Pew, different states included different 
costs. Third, one cannot assume that what is average is necessarily appropriate. Fourth, 
the data is a few years old. For these reasons, I sought a more Arizona-appropriate and 
recent benchmark. 

The best benchmark I could identify to judge the adequacy of ADC’s health care 
spending is Arizona Health Care Cost Containment System (AHCCCS), the state agency 
which insures Arizonans on Medicaid. Of the various subgroups covered by AHCCCS, 
the subgroup of individuals whose income is between 0 - 100% of the Federal Poverty 
Level is the population that almost perfectly matches the ADC population 64 , except as 
noted below. With AHCCCS’s help, I calculated what the approximate cost of healthcare 
would be for the ADC population if that care were being paid for by AHCCCS in the 


64 The appropriateness of using the AHCCCS 0-100% FPL population (also called “Childless 
Adults” by AHCCCS) as the population of comparison is based on: (a) my knowledge of prison 
populations; (b) a meeting and subsequent communications Mr. Pratt and I had with a financial 
expert within AHCCCS; and (c) information from the lead Corizon Release Planner. Release 
Planners are health care staff who assist ADC residents with their transition back to the 
community, including assisting them to obtain health care insurance. The Release Planner 
confirmed that it is rare (~1 in 400 or 450) for a releasing ADC resident to have income at higher 
than 100% FPL. 
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community. The input variables I used and the resultant calculations are contained in the 
box below. 65 


What the Annual Health Care Costs of ADC Population Would be Using AHCCCS Rates 

Component of Cost Annual Costs 

AHCCCS capitation rate for non-SMI (Seriously Mentally Ill) childless adult males, 

CY 2019 = $558 per month * 12 months = $6,696 per year 
ADC non-SMI male population, April 2019 = (33,967-2,034 SMI) * 89.8% males = 28,675 

Subtotal = $6,696 * 28,675 = $192,007,000 

AHCCCS capitation rate for non-SMI childless adult females, 

CY 2019 = $675 per month * 12 months = $8,100 per year 
ADC non-SMI population, April 2019 = (33,967-2,034 SMI) * 10.2% females= 3,257 

Subtotal = $8,100 * 3,257 = $26,381,000 

AHCCCS capitation rate for SMI childless adult, 

CY 2019 =$2,020 per month * 12 months = $24,241 per year 
ADC SMI population, April, 2019 = 2,034 

Subtotal = $24,241 * 2,034 = $49,306,000 

AHCCCS supplement for emergency dental care = $28 per year 
ADC population, April 2019 = 33,967 

Subtotal = $28 * 33,967 = $951,000 

AHCCCS supplement for pregnancy = $175 per month of pregnancy 

Pregnancy-Months = 25 pregnant women in ADC in 2018 on an average day * 12 mos. = 300 pregnancy-months 

Subtotal = $175 * 300 = $52,500 

AHCCCS supplement for delivering a baby = $5,500 per delivery 
ADC births, 2018 = 31 

Subtotal = $5,500 * 31 = $170,500 

Total Annual Health Care Cost $268.868.000 


Based on these calculations, in my opinion, health care in ADC suffers from severe 
underfunding; the gap between what ADC is currently spending on health care and what 
it should be spending on health care is at least $74 million (see box below). 


65 The monthly rates for non-SMI adult males and females I use in this calculation are $558 and 
$675, respectively. AHCCCS doesn’t report rates by gender. They report a combined monthly 
rate for all adults of $622. Because the gender composition of AHCCCS and ADC differ, this 
latter rate needs to be adjusted for gender by calculating separate rates for males and females 
from the AHCCCS data. To impute these rates I used AHCCCS’s reported rate of female insured 
for 2018, which was 54.2% 

(https://www.azahcccs.gov/Resources/Downloads/PopulationStatistics/2018/Apr/AHCCCS De 

mographics.pdf) , and the relative cost of health care for females compared to males reported by 
the federal government (for the whole population, not limited to Medicaid recipients) in 2014: 
health care for females was 21% costlier than for males, (https://www.cms.gov/research- 
statistics-data-and-systems/statistics-trends-and-reports/nationalhealthexpenddata/nhe-fact- 

sheet.html) . 
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Spending level for ADC population using AHCCCS rates $268,868,000 

with adjustment for gender 

Current ADC spending level (as of July 1, 2019) $195,160,000 

Conservative estimate of spending gap, i.e. without $73,708,000 

adjustment for unmeasurable factors described below 


There are some potential limitations to this comparison of AHCCCS costs to ADC costs. 
First, the AHCCCS costs do not include most over-the-counter medication. By contrast, 
ADC pays for many over-the-counter medications which individuals would purchase out- 
of-pocket if they were living in the community. However, if this factor were included in 
the calculation, the spending gap would be larger. Second, AHCCCS costs do not include 
dental costs other than for dental emergencies. ADC is constitutionally bound to, and 
does, provide medically necessary dental care beyond just emergencies. However, if this 
factor were included in the calculation, the spending gap would be larger. Third, my 
calculation does not account for individuals with federally-defined disabilities. AHCCCS 
pays a higher capitation rate for such individuals ($13,200-$14,000 per year vs. $7,464 
per year). I was not able to include this factor in my calculations because the number of 
ADC patients who meet the federal definition of disabled is not currently available. 
However, if this factor were included in the calculation, the spending gap would be 
larger. Fourth, the gender-specific costs of health care I used had to be imputed from the 
costs reported by the federal government. Those costs were for the U.S., not Arizona, and 
were for the whole population, not just those on Medicaid (i.e., those similar to the 
AHCCCS population). While my imputations are based on reasonable assumptions, if 
one were to use the AHCCCS rates for males and females, the spending gap might be 
smaller or greater. Fifth, my calculation does not take into consideration any differences 
in cost due to differences in age distribution between AHCCCS and ADC. I compared 
age distributions for adults (18+ years old) using data from AHCCCS 
(https://www.azahcccs.gov/Resources/Downloads/PopulationStatistics/2018/Apr/AHCC 

CS Demographics.pdf) and ADC (provided by ADC’s Research Office and adjusted by 
them to match the age groups used by AHCCCS). The comparison is shown in the table 
below: 


Aged Distribution of Individuals 
Covered by AHCCCS vs. Residents of ADC 


Age 

AHCCCS 

ADC 

18-21 

10.7% 

2.8% 

22-64 

79.0% 

94.7% 

65-79 

7.4% 

2.2% 

80+ 

2.9% 

0.1% 
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There are differences. The AHCCCS has a greater percentage of individuals in the lowest 
age bracket than ADC. One would expect these patients to be less costly. On the other 
hand, AHCCCS has a greater percentage of individuals in the two highest age brackets 
than ADC. One would expect these patients to be more costly. ADC has a greater 
percentage of individuals in the age bracket 22-64 than AHCCCS. We would expect the 
cost of care for those at the low end of this age bracket to be significantly less than the 
cost of care for those at the high end. Depending on whether the distribution of 
individuals in this bracket is similar or different between ADC and AHCCCS, the cost of 
care in ADC’s population could be greater, smaller, or the same as the cost of care in 
AHCCCS’s population. Thus I am unable to determine how the distribution of ages might 
affect a comparison between the two populations. Sixth, in a section below (see Salary 
Limitations and Delays in Fulfilling Provider Requests for Specialty Services - 
Accessing Community Specialists) I describe two factors which drive up costs for ADC: 
the salaries to employed professionals, and the fees paid to community specialists. Both 
these factors are different in corrections than they are in the community, and thus are not 
reflected in AHCCCS’s rates. Unfortunately, one often must pay health care 
professionals (especially providers) a premium to work in a correctional setting or care 
for residents of a correctional facility. I did not include the costs associated with these 
two factors in my spending gap calculation because it is difficult for me to quantify their 
marginal costs. However, if these factors were included in the calculation, the spending 
gap would be larger. Seventh, and finally, my calculation does not take into account the 
profit margin enjoyed by the health care vendor (Corizon prior to July 1, 2019; Centurion 
as of July 1, 2019). I was unable to include this factor in my calculations because neither 
company’s profit margin is publicly available. Based on my knowledge of the industry, it 
is likely in the range of 6 - 9%. Adopting an even more conservative estimate of 5%, the 
profit margin would be close to $10 million. This amount reduces the dollars ADC is 
spending on actual health care. In other words, while ADC is writing a check for $195 
million, it is only spending $185 million on health care. Thus the effective spending gap 
is closer to $84 million. 

In summary, in my opinion, the severe level of underfunding of health care services at the 
ADC is the single most significant barrier to compliance with the PMs in this case. At a 
minimum, the gap between what it costs to take care of this population according to 
AHCCCS rates and what ADC spends, is at least $74 million. This amount is likely 
conservative because I was unable to include a number of factors in my calculations, such 
as the cost of: over-the-counter medications; non-emergency dental care; care of patients 
with federally-defined disabilities; age-based care; marginal cost for health care 
professional salaries and fees when working with residents of a prison; and the vendor’s 
profit. With the exception of age, the effect of inclusion of any of these seven factors in 
the calculation will only result in a larger calculated spending gap. Accounting for 
differences in the age distribution in AHCCCS compared to ADC could result in the 
spending gap to be larger or smaller than I estimate. However, even if adjustment for age 
caused the spending gap to be smaller, it is unlikely that it would overshadow the effects 


94 


Case 2:12-cv-00601-ROS Document 3379 Filed 10/04/19 Page 95 of 138 


of the other six factors I could not account for. In other words, my estimation that the 
spending gap is at least $74 million, is robust. 

Recommendation 51: 

ADC’s spending level should be significantly increased immediately. It would be 
reasonable to limit the initial increase to 50% of the minimal gap (i.e., $35 million), and 
then make adjustments over the subsequent 6-12 months. 

I do not have enough information to provide more specific recommendations on how 
those additional funds should be spent, but I can provide some general recommendations 
about where some of those additional funds should be allocated: 

• Staffing levels need to be increased. 

• The mix of staff (e.g. physician vs. mid-level provider 66 ) needs to be 
reconfigured. 

• Salaries may need to be increased. 

• Community specialist fee payments need to be increased. 

• The functionality of eOMIS needs to be improved. 

Each of these is addressed in the following sections. 

Plaintiffs concur. Defendants state that staffing levels cannot be increased “as the 
Stipulation does not allow it.” They also offer, “Defendants do not take a position on Dr. 
Stem’s findings or recommendations here. Funding is a legislative matter... .None of the 
remaining PMs are systemically failing across the complexes. Compliance at most 
facilities across the measures demonstrates that compliance can be achieved and speaks 
against underfunding as the cause.” 

Where to Allocate Additional Funds: Increase Staffing Levels 

Issue: 

Staffing levels need to be increased. With the exception of psychiatric prescribers (see 
below), there are no generally accepted formulas for calculating minimal staffing levels 
for correctional facilities. The soundest advice from both the National Commission on 
Correctional Health Care and the American Psychiatric Association (APA) is that staffing 
is adequate when necessary tasks are being carried out. 

There are five approaches which can be used to inform the adequacy of staffing levels at 
ADC: First, in the very narrow niche in which a formula exists, a formula for psychiatric 
providers (psychiatrists and psychiatric nurse practitioners) can be applied. The APA 
makes a very general recommendation of 1.0 FTE psychiatrist for every 50 patients in a 
residential mental health unit and 1.0 FTE psychiatrist for every 150-200 patients with 


66 Mid-level providers are nurse practitioners and physician assistants. 
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serious mental illness (SMI) in the general prison population (i.e., not in a residential 
mental health unit) on medication, but is cautious in noting that these must be modified 
on a case-by-case basis. Applying this formula to ADC, the number of psychiatric 
providers was much too low to meet the need under the Corizon contract, but may be 
within a reasonable range under the Centurion contract which began 7/1/19, depending 
on rough assumptions 67 . 

Second, one can compare budgeted positions to filled positions. This method rests on (a 
reasonable) assumption that prison systems - in fact any organization - rarely over¬ 
budgets for personnel. Thus if the organization budgeted for X positions, but fewer than 
X positions are currently filled, it is reasonable to conclude that the organization is 
understaffed to at least that degree. According to the May 2019 monthly staffing report 
produced by Corizon, 1,017.75 positions were budgeted for ADC healthcare, but only 
844.70 positions (83%) were filled. Thus, it is reasonable to conclude that ADC was 
understaffed at that point by a minimum of 17%. This is a very large gap. 


67 This is based on the following calculation as of July, 2018. ADC has approximately 390 
patients in residential mental health units (MH-4 level) who use the services of a psychiatrist. 
(APA’s formula is predicated on the assumption that most patients in residential mental health 
units are on psychotropic medications. At ADC, that is not the case. Based on a small sample 
analysis ADC conducted for me, 25% of the patients in residential mental health are not on 
psychotropic medications and thus do not typically see a psychiatrist. Thus the number used here 
was obtained by discounting the total number of patients in residential mental health units in 
July, 2018 - 512 - by 25%.) According to APA’s formula, they would be cared for by 7.8 FTE 
providers. According to APA’s formula, they would be cared for by ADC has 77 MH-5 level 
patients (who, generally all are cared for by a psychiatrist). According to APA’s formula, they 
would be cared for 0.4 to 0.5 FTE. ADC has approximately 1,600 MH-3A level SMI patients in 
general population, of which ADC officials estimate half are on medication. According to APA’s 
formula, they would be cared for by 4 to 5.5 FTE providers. APA’s formula does not address 
provider staffing for non-SMI, non-residential mental health patients who also must be on the 
psychiatric providers’ caseload. There are 6,068 such patients in ADC (5,403 MH-3B; 665 MH- 
3C). Assuming - conservatively - that such patients only require 50% of the time commitment of 
SMI patients, ADC would require another 15.2 to 20.2 FTE providers. Finally, on an average 
month, there are 197 patients who recently had their psychotropic medications discontinued 
(MH-3D level) and therefore require an encounter with a psychiatrist. A (typical) visit length of 
approximately 15 minutes for each of these 197 encounters would require approximately 0.3 
FTE (197 visits/month x 0.25 minutes/visit -f 4 weeks/month -r 40 hours/week). In total, the 
formula would predict the need for 23.7 to 29.2 FTE (7.8 + 0.38 to 0.5 + 15.2 to 20.2 + 0.3). 
Under the Corizon contract, ADC was staffed with 21.5 FTE mental health providers. As of 
7/1/19, under the Centurion contract, ADC should be staffed with 31.0 FTE mental health 
providers. This is above the 23.3 to 28.3 FTE range suggested by APA’s formula, but only if the 
very conservative 50% assumption above is correct. 
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Third, one can compare the budgeted staffing levels at two points in time relative to the 
population sizes at those two points. I examined the staffing levels at ADC at June, 2012, 
to the Spring of 2019. The results are shown in the table below. 


Point in Time 

June, 2012 
May, 2019 


Population 

33,638 

33,967 (1% t) 


Budgeted Staffing Level 

1,139 FTE 
1,018 FTE (11% |) 


Thus it is reasonable to assume that ADC healthcare is additionally (i.e., in addition to the 
17% above) understaffed by a little over 11%. 

Fourth, one can use professional judgment. This had limited application and is most 
useful at the extremes of staffing levels. The nature of my present review for the Court 
was not designed to provide an in-depth judgment on staffing levels. However, I am able 
to comment on two staffing conditions I noticed during my work. 

At Loewis Complex the sole physician (1.0 FTE) is responsible for caring for all the 
patients in the 13-bed infirmary (IPC). That physician also functions as the facility 
medical director and supervises seven nurse practitioners, each of whom carries a full 
load of about 750 patients. The role of caring for the patients in the IPC, by itself, 
requires at least 0.5 FTE physician 68 , and may require more depending on the acuity of 
IPC patients. Given the population size of the Ixwis Complex, the administrative role of 
facility medical director requires at least 0.5 FTE. And supervision of seven nurse 
practitioners with full loads of complex general medicine patients (i.e., the types of 
patients found in a facility like Lewis) ordinarily requires about two physicians, assuming 
that those physicians have sufficient protected time carved out to provide supervision. 

The seven nurse practitioners are spread out across a 260 acre complex, which 
significantly decreases the efficiency of physician supervision. Finally, while well-trained 
experienced nurse practitioners can safely care for many patients in a prison caseload, 
there are typically many patients in a prison like Lewis who are so complex that direct 
care management from a nurse practitioner is inefficient. Based on all these factors, I 
conclude that the Lewis Complex is dangerously understaffed in terms of physicians. 

This is corroborated by the fact that in the recent past, 3.0 FTE physicians were stationed 
at Lewis, compared to 1.0 FTE now. 

At Perryville I interviewed several mental health professionals. Among them, four have 
caseloads of 400-500 patients and one has a caseload of 300 patients. 69 Given the nature 
of the patients they see, caseloads should be closer to 100-150 patients. As discussed 


68 This and the subsequent staffing level expectations in this paragraph are based on my 
experience. 

69 This may be due to maldistribution of caseloads (as there were also some professionals 
carrying very small caseloads) in which case adequate supervision may remediate the problem. 
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elsewhere in this report, there is evidence that some encounters with mental health 
professionals during watch and non-watch encounters are very short. As also explained 
elsewhere, there may be legitimate reasons for short encounters. However, to the extent 
that some of these short visits are driven by time pressure 70 , that would be evidence of 
understaffing. Thus in one, and possibly two settings, based on my professional judgment 
(and, for the Lewis venue, based also on recent budgeted staffing), ADC is significantly 
understaffed. 

Fifth, one can perform a formal staffing analysis. The analysis uses a combination of the 
four approaches above along with other techniques, including time-motion studies, 
interviews with managers and front line workers, observation of work flow, and, most 
importantly, a review of key tasks that are being done poorly and/or with unacceptable 
delays to calculate the marginal increase in staffing required to correct the deficiencies. 

Recommendation 52: 

ADC should conduct a staffing analysis and then implement staffing changes 
accordingly. Because a proper staffing analysis can take a few months to complete, ADC 
should immediately make staffing adjustments that are obviously needed, such as the two 
I describe above. The analysis could be conducted by ADC alone, or with assistance from 
an external expert. If an expert is used, however, he/she/it should be independent of any 
for-profit correctional health care service provider. 

Plaintiffs concur. Defendants state that staffing levels cannot be increased “as the 
Stipulation does not allow it.” They also offer, “Defendants do not take a position on Dr. 
Stem’s findings or recommendations here. Funding is a legislative matter. Defendants 
note, however, that Dr, Stem recommends retiring/terminating 786 of the remaining 966 
PMs. Only 180 PMs would remain. None of the remaining PMs are systemically failing 
across the complexes. Compliance at most facilities across the measures demonstrates 
that compliance can be achieved and speaks against underfunding as the cause.” 

Where to Allocate Additional Funds: Reconfigure the “Mix” of Staff 

Issue: 

There are some clinical tasks at ADC that are performed of by professionals in two 
different disciplines or two levels of professionals within the same discipline. While I 
was not charged to assess the overall safety of patient care, there are five such 
“profession pairs” at ADC that deserve discussion based on my review, either because 
one of the two professionals should not perform certain tasks or because, while both 


70 I collected conflicting evidence of time pressure from Corizon mental health professionals. 
One former employee at Phoenix, who also testified in the case, told me of extreme time 
pressures. Some others at Perryville also reported time pressure. On the other hand, current 
professionals at Eyman and Phoenix told me they were not under time pressure. 
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professionals may perform the task, the ratio of the number of staff at each level (“mix”) 
may be so disproportionate as to be unsafe for patients. 

RN/LPN I found examples (see the discussion of Patient 23 in Part II of this report, PM 
37) of LPNs performing tasks outside their safe and legal scope of practice, most notably 
assessing patients and designing nurse care plans independently. This is dangerous. Such 
tasks should only be performed by an RN, a mental health clinician, or a provider, as 
appropriate. 

Physician/Mid-Level Provider In the previous section (Increase Staffing Levels) I 
describe the ratio of only one physician supervising seven nurse practitioners at one of 
the complexes. This is a dangerous mix. Not only should mid-level providers have an 
adequate level of collaboration with physicians as they care for the patients within their 
ability to handle, but given the level of disease in a prison population, there are some 
patients who are so complex as to require their direct care to be provided by a physician 
rather than a mid-level provider. This is well demonstrated by the frequency with which I 
encountered episodes of patient care in which clinical decisions made by mid-level 
providers were clinically unsound and dangerous. Thus the ratio of physicians to mid¬ 
level providers needs to be increased. 

Licensed/Unlicensed Mental Health Clinicians A number of mental health-related PMs 
required tasks to be performed only by licensed clinicians. However, a number of 
complexes have been found Noncompliant due to use of unlicensed clinicians. 

Provider/RN It is within the legal scope of practice for RNs to see patients presenting 
with new (“episodic”) complaints independently. However, at ADC, RNs are charged 
with seeing almost all such cases. In a number of examples I reviewed, it was clear to me 
that such care was well beyond the capabilities of the RN and sometimes dangerous. It is 
unreasonable to expect that RNs can safely independently assess the broad scope of 
chnical problems prison patients present with, even armed with the nursing care guides 
(Nursing Evaluation Tool; NET) they are given. 

Board Certified Physician/Non-Board Certified Physician 

Board certification is granted to physicians in their specialty (e.g. internal medicine, 
family practice) after having completed a residency and passing an examination. While 
board certification is not a legal requirement for physician licensure, it helps ensure a 
certain level of patient safety. Further, among the board certifications, the primary care 
specialties (internal medicine and family practice) are the most relevant to providing 
primary care in a prison. My review does not assess in depth the overall quality of care 
delivered by physicians in ADC; however, if it is determined that unsafe medical 
decisions are being made by non-board certified physicians or physicians certified in non¬ 
primary care specialties, then increasing the mix of primary care board-to-non-board 
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certified physicians (or simply limiting practice at ADC to board certified physicians in 
primary care), may be indicated 71 . 

Recommendation 53: 

ADC must increase the number of RNs relative to LPNs, physicians relative to mid-level 
providers, and licensed relative to unlicensed mental health professionals. Increase in the 
number for RNs relative to providers may or may not be necessary, depending on the 
model of care adopted. If, for example, ADC adopts a model where RNs provide more 
routine chronic care (freeing up providers to provide more episodic care), the mix may 
not need to change. Finally, though examples in Part IV of my report suggest it, my 
review does not assess whether overall care provided at ADC via non-board certified 
physicians is dangerous. If it is determined to be so, then all physicians should be board 
certified, and that board certification should be in a primary care specialty. Each of these 
increases is accompanied by increased cost. 

Plaintiffs concur. Defendants do not concur with the findings and recommendations 
related to mental health staff (“Licensed/Unlicensed Mental Health Clinicians”), noting, 
“Licensed mental health clinicians are required for PMs 73, 74, 94, and 95. Florence was 
in substantial noncompliance for a few months last year but has been compliant since. 
There is no systemic failure with current staffing. There are 72 licensed clinicians and 13 
unlicensed.” 

To paint a fuller picture, the Florence complex was Noncompliant with PM 94 for five 
consecutive months at the end of 2018 (47%, 27%, 0%, 7%, 67%) and Noncompliant 
with PM 95 for four consecutive months toward the end of 2018 (70%, 55%, 40%, 50%). 
In addition, the Perryville complex was Noncompliant with PM 74 in December, 2018 
and January, 2019 (71% and 83%, respectively). Nonetheless, as Defendants’ comment 
suggests, the problems caused by use of unlicensed mental health clinicians, as measured 
by PMs 73, 74, 94, and 95, may well be on the road to recovery, but it is too early to say 
for certain. 

Where to Allocate Additional Funds: Increase Salaries 

Issue: 

Some current staffing vacancies are due to compensation which is not competitive 72 . If 
ADC self-operates health care, it will need the ability to adjust wages to recruit suitable 
candidates. The correctional health care market is a unique niche where competitiveness 
of salaries cannot be determined simply by surveying community salaries. ADC may 

71 This does not apply to physicians whose task is to provide non-primary care, such as 
gynecology or psychiatry. 

72 I make this assertion notwithstanding the conclusion of the Advisory Group (Doc. 2940) that 
salaries are competitive. The reason is that the Advisory Group opinion was based on 
comparison to salaries in the community. The prison and community workplace markets are not 
the same for at least the three reasons explained in the text. 
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need to offer higher salaries due to corrections-specific factors, most notably: 1. Many 
professionals view corrections as a less desirable work place. 2. Some prisons are located 
in locations that are not close to population centers (e.g. Florence). So a salary premium 
is required to entice health care professionals to move to these more remote locations or 
spend time commuting. 3. The difficulty accessing the workplace adds uncompensated 
time to the workday. This is particularly evident at a facility like Lewis, where employees 
may have to allow an extra hour to get to and from their parked vehicle and their work 
stations, due to a large campus and lengthy security measures. 

Currently, R2-5A-401 Salary Plan, (B) Alternative Salary Plan allows for special salary 
adjustments: “The Director [of the Department of Administration (DOA)] may establish a 
special salary plan or pay practice determined to be the prevailing practice in the labor 
market and in the best interest of the state.” Such a process is not nimble enough to 
accommodate the urgent and frequent needs of the ADC, nor does the benchmark of the 
“prevailing practice in the labor market” meet the special needs of the corrections labor 
market, as described above. 

Recommendation 54: 

Until such time as ADC is able to establish and maintain a full work force, R2-5A-401 
Salary Plan, (B) Alternative Salary Plan should be suspended; the authority to create an 
alternative salary plan should be vested in the Director of ADC, rather than the Director 
of DOA, and the salary plan should be based on his/her best professional judgment. 

Plaintiffs concur. Defendants take no position. 

Where to Allocate Additional Funds: Increase Community Specialist Fee Payments 

Issue: 

In 2009 the Arizona Legislature instructed ADC to cap payment to community specialists 
at the level adopted by AHCCCS. (Arizona House of Representatives, “HB 2010: 
Criminal Justice; Budget Reconciliation,” 2009) Based on my conversations with 
Corizon staff who are responsible for trying to schedule appointments for ADC residents 
with community specialists, a conversation with the office manager of a community 
specialist who elected to discontinue seeing ADC patients, and my experience working 
with community specialists providing care to jail and prison residents, it is my opinion 
that this markedly lower payment rate is a major factor in reducing the number of 
specialists available to see ADC patients. This, in turn, contributes to delays ADC has 
witnessed in patients receiving specialty services, as measured by PMs 50 and 51. 

Though unfortunate, it is not unexpected that community specialist behavior is sensitive 
to payment rates when dealing with prison patients. First, many of them find having 
prison patients, dressed in bright jump suits, restrained in arm, waist, and leg shackles, 
accompanied by officers with weapons, “bad for business.” Second, managing these 
patients, with the attendant difficulties interfacing with the prison system, is more 
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difficult and time consuming. For example, appointments are more likely to be cancelled 
because of custody/transportation problems. Third, many of them believe that prisoners 
are more litigious. Thus, while AHCCCS payment rates may not be unreasonable for 
patients living in the community, they fail to take into account the real market forces 
which make them unreasonable when treating a prison population. 

Recommendation 55: 

I recommend that the Legislature’s instruction to ADC to cap payment to community 
specialists at the level adopted by AHCCCS (Arizona House of Representatives, “HB 
2010: Criminal Justice; Budget Reconciliation,” 2009) be rescinded or overridden. ADC 
should be allowed to pay community specialists at the rate necessary, based on market 
forces, so that it can provide medically necessary care to its patients and provide that care 
in a timely manner. 

Plaintiffs concur. Defendants take no position. 

Where to Allocate Additional Funds: Electronic Health Record (EHR) Improvement 

Issue: 

ADC uses an EHR called eOMIS. An EHR has some advantages over a paper medical 
record and has the potential of being a more powerful and safer way of recording patient 
health information. eOMIS achieves some of those benefits. For example, the record is 
available to anyone at any time from any location, and entries are always legible. 
However, if poorly designed, an EHR can present challenges to the user. eOMIS presents 
such challenges. While I am unable to tie any specific eOMIS design flaws to a specific 
patient risk, taken as a whole, the design flaws in eOMIS make it more difficult for health 
care staff to do their jobs, including those tasks measured by the PMs, as well as other 
tasks which reduce the risk of serious harm. 

The following are just a few examples of eOMIS’ poor design for users. 

• Scanned documents (documents which originally existed in paper format and 
must be imported into the EHR) are filed according to the date they are scanned (a 
function of when the clerk happens to get around to scanning) as opposed to the 
relevant date, i.e., the date of the event addressed in the document. This makes it 
difficult for a user to quickly find a document he or she needs for patient care. 

• When attempting to view documents in the Scanned Documents/Photos section of 
the EHR, eOMIS is programmed to present an arbitrarily partial list of recently 
scanned documents (however, the user does not know that this is a partial list 
without conducting further investigation). To view a complete list in 
chronological order, the user must enter three additional commands. To select a 
scanned document to view, the user must then enter three or four additional 
commands. This is time consuming. 
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• eOMIS does not allow a user to open more than one page of a patient’s record 
simultaneously (with the exception of scanned documents). It is commonly 
necessary for users of a patient’s medical record to “flip” back and forth among 
several related documents in order to fully understand the clinical issue at hand. 
For example, when reviewing a patient’s last 72 hours in an IPC, a provider 
would typically need to chronologically track nursing notes, the patient’s vital 
signs, and any blood tests, looking at all three sources of information 
simultaneously. In eOMIS, the user must close one document to view the next, 
and if that document raises a question that requires re-examining the first 
document, the process repeats itself. This is a frustrating and time consuming 
exercise. 

• eOMIS does not include a function that allows users to view test results 
sequentially. One of the most important evaluations clinicians conduct on tests is 
to view the trend of results of the same, or related tests, over time. Most modem 
EHRs even allow clinicians to view the results of a given test graphically, over 
time. In eOMIS, the user must click on each individual test result, and close that 
result before opening the next result. This makes it very difficult to view the 
temporal course of a condition, for example how well a patient is responding to 
treatment for diabetes. 

• eOMIS does not appear to have failsafe mechanisms to alert users and 
administrators when there are delays in completion of scheduled tasks. For 
example, if a provider orders a blood test, neither the provider nor an 
administrator receives an alert if the test result does not return within a specified 
length of time (which could mean there was a problem with obtaining of the 
sample, processing of the sample, or communication of the result from the 
laboratory to the prison). As a result, scheduled patient care tasks, such as 
diagnostic tests, are not performed or are performed late. 

• It is very difficult to “thumb through” successive encounters of a particular type. 
For example, imagine a psychiatrist wishing to view successive psychiatric 
encounters for Patient 28. On the “Encounter” tab in eOMIS, she would click on 
“Type” of encounter, to organize the encounters so that psychiatrist encounters 
(“MH - Psychiatrist - Scheduled”) are grouped together (in chronological order). 
She would then have to scroll down approximately 1,320 encounters to find this 
group of encounters among the “M”s. Once there, she would click on the first 
encounter of interest and read it. To view the next successive encounter, she 
would click on the “return” button to return to the “Encounter” list which she 
previously organized by “Type” of encounter. However, eOMIS automatically 
returns her to the top of the “Encounter” list. So now she needs to manually scroll 
down 1,319 encounters once again to find the next successive psychiatric 
encounter. She must do this for each successive encounter she wishes to review. 
To make her task even more daunting, “MH - Psychiatrist - Scheduled” may not 
be the only psychiatric encounter the patient has had. He may have had scheduled 
or unscheduled encounters with the psychiatrist, the psychiatric nurse practitioner, 
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or the psychiatric mid-level practitioner interspersed chronologically among each 
other. Each of these six encounters is a different “Type” of encounter, grouped 
separately in the list of encounter (which, in total, is 1,600 visits, or computer 
rows, long). So to read one successive encounter after the next in chronological 
order would also require her to also jump among these six groups. 

• Many pages of a patient’s medical record are littered with useless or 

incomprehensible information, making it exceeding difficult for care givers to 
find the relevant information they are looking for and to understand the rest. The 
following screen shot from the first part of the Problem List of Patient 43 is a 
typical example. All these words and numbers obscure a rather small amount of 
necessary information that a provider or nurse needs to be able to glean quickly: 


about 10 chronic conditions from which the patient suffers. 



Recommendation 56: 

ADC needs to identify the inefficient and dangerous components of eOMIS and engage a 
programmer to redesign it. Front line users of the medical record should be key 
informants of this process. 

Plaintiffs concur. Defendants note that they are in the process of making improvements to 
eOMIS. 


Privatization of Health Care Services 

Re-establish Self-Operation of Health Services 

Issue: 

In 2009 the Arizona Legislature instructed ADC to privatize correctional health services. 
(Arizona House of Representatives, “HB 2010: Criminal Justice; Budget Reconciliation,” 
2009) In my opinion, privatization has not served, and will continue to not serve, ADC 
well. It is, after insufficient funding of health care, the second greatest barrier to 
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compliance with the PMs in this case. There are several mechanisms by which 
privatization has been, and will continue to be, a barrier to compliance. 

1. Increased Cost 

As explained earlier, privatization of correctional health care costs the state more than 
self-operation. If health care were over-funded by an amount equal to the vendor’s profit 
margin, cost would not be a barrier. However, it is clear from my analysis earlier, that 
health care is not over-funded. Therefore, at least $10 million of state expenditures - a 
conservative estimate of a vendor’s profit margin - are not currently being applied to 
improving health care and compliance with PMs. 

The costs associated with privatization are not limited to profit margin. There are at least 
three other costs drawing funds away from PM compliance. First, privatization drives 
duplication of staffing and services. The vendor has monitors to make sure they comply 
with the PMs and other requirements of the contract; ADC has monitors to do the same. 
The vendor has a contract manager and statewide medical director; ADC has a contract 
overseer and a medical director. The vendor has staff to follow and manage the costs of 
the contract; ADC has staff to follow and manage the costs of the contract. The vendor 
has lawyers to draw up, modify, and deal with issues related to the contract; ADC has 
lawyers to do the same. Second, there is considerable cost (aside from attorneys’ costs) to 
develop and issue an RFP, vet bidders, and negotiate (and re-negotiate) a contract. Third, 
there is considerable cost associated with transition from vendor to vendor. I am observed 
this as the contract transitioned from Corizon to Centurion. ADC, Corizon, and Centurion 
staffs spent hundreds of hours in this endeavor. ADC pays, not only directly for the time 
of its employees who assist with transition, but indirectly for the time of Corizon and 
Centurion employees. 

In the previous section I discuss the large spending gap between what ADC should be 
spending on its health care operation based on AHCCCS benchmarks, and what it is 
spending. Switching back from privatization to self-operation would, in effect, 
immediately reduce that spending gap by shifting funds ADC is currently spending on the 
non-value-added parts of contract expenses (vendor profit margin, duplication of 
services, cost of issuing a contract, cost of transition) to patient care. 

2. Dangers at Transition 

The transition from one vendor to another is a highly complex event. In any health care 
organization, such events pose a high risk for errors, and therefore a risk of substantial 
harm to patients. Thoughtful planning can reduce those risks, but they cannot be 
eliminated. Under privatization, such risk taking is, by design, destined to recur whenever 
a vendor is changed. The following is an example of actual risk due to the recent vendor 
transition. Patient 8 has breast cancer for which she is being followed by an oncologist. 
The patient complained of a non-healing lesion inside her nose. The oncologist 
recommended that the patient be referred to an Ear Nose and Throat (ENT) specialist for 
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a biopsy; in this patient, such a lesion could be cancer. On 5/9/19, a provider requested 
the referral from the ENT specialist with whom Corizon had a contract. Because the ENT 
specialist’s schedule was very full, the patient was placed on a waiting list. On 7/16/19 
(16 days after the start of the Centurion contract) the ENT specialist’s office called the 
patient’s complex and informed them that they no longer see patients under the Centurion 
contract. Centurion identified a new ENT specialist and requested a consultation. As of 
8/3/19, almost three months after the referral was requested 73 , the patient still does not 
have an appointment date. 

3. No Marginal Benefit 

One of the three good reasons for privatizing correctional health care is to provide the 
expertise required to operate a correctional health care service to an organization lacking 
such expertise. After having spent scores of hours working with staff of ADC’s health 
care Monitoring Bureau over the past few months it is clear to me that ADC already has 
the requisite leadership, clinical expertise, and talent, in house, to safely operate a health 
care service. 

The second good reason for privatizing correctional health care is to benefit from 
economies of scale brought to bear by a large vendor. A major one of such economies is 
the purchasing of pharmaceuticals. However, in 2019, states have viable options to 
contract directly with pharmaceutical vendors or to work with interstate governmental 
cooperatives, to enjoy the same economies of scale. 

The third good reason for privatizing is to be able to pay a salary that the union contract 
or civil service rules would otherwise prevent. Civil service rules in Arizona do limit 
salaries, and, as discussed above (see Increase Salaries) are a contributing factor to the 
barriers to compliance. However, (a) the vendor has not exploited its lack of salary 
limitation to make salaries competitive enough so that positions are filled, and (b) while 
under Arizona’s civil service rules it is difficult to exceed preset salary levels, it is not 
impossible; in the section Increase Salaries, above, I recommend changes to the rules 
which can make it easier. 

Thus privatization has not brought ADC any advantages that it is not already ably poised 
to achieve on its own. 

4. Lack of Maneuverability 

The changes to health care operations that managers must make to respond to day-to-day 
emergencies and developments in the best of times, no less when undergoing litigation, 
require an organization to be flexible and nimble. Intercalating a vendor into the mix all 
but prevents that maneuverability. An excellent example presented itself during one of 
my tours. A change in circumstances required replacing an LPN position with an RN 
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The consult was requested as “Routine,” thus was required to have been completed by 7/9/19. 
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position. Under self-operating conditions, this change would have taken no more than 
five minutes of work and a stroke of a pen. Instead, because it required interfacing among 
ADC, attorneys to modify the contract, and the vendor, it consumed four hours of the 
Monitoring Bureau Director’s time as well as an unknown amount of time for the other 
Parties. 

The impairment to maneuverability caused by privatization of ADC health care services 
can be seen in another example regarding telemedicine (TM). TM is a powerful tool that 
could help solve some of the challenges ADC faces in accessing specialists (PMs 48, 50, 
51). It is my understanding that Corizon twice tried to forge relationships with individuals 
or organizations to provide more TM services 74 to ADC patients, but the candidate TM 
providers declined to contract because of their reluctance to invest all the resources 
necessary to set up the service with a company that might, due to the nature of the 
RFP/contracting cycle, not be around long enough to make the venture worthwhile. 

5. Recruitment Challenges 

Based on my conversations with vendor employees (regarding their own experiences and 
those of colleagues who no longer work at the prisons), it seems that many prefer to work 
as employees of the state than of a private company. The two reasons cited were better 
benefits and a personal feeling of greater fulfillment as an agent “of the people” doing 
good work for society. According to anecdotal reports from individuals who were at 
ADC prior to privatization, vacancy levels were significantly lower then. 

6. Poor Track Record 

Finally, the empiric evidence shows that the privatization experiment has a high risk of 
failure. ADC has just entered its third contract for health services since first privatizing in 
July, 2012. The first two contracts were fraught with problems. Thus even if the third 
contract meets all expectations, private contracting has a success rate of 33% at ADC. 

In summary, in my opinion, privatization of health services at ADC is an important 
barrier to compliance with PMs and other risks to patient safety which I will describe in 
Part IV of my report. 75 


74 Corizon provided some psychiatric services by TM. 

75 Privatization causes another noteworthy problem: patient financial burden. Corizon has failed 
to pay some community providers for healthcare delivered under their contract with ADC. For 
example, Patient 32 received a bill for $12,371 in May of 2019 for a hospitalization which 
occurred nearly a year earlier and for which Corizon had not yet paid the hospital. When the bill 
is not paid, some providers seek payment directly from the patient, including engaging the 
services of a collection agency. Aside from the stress and annoyance of pursuit by the collection 
agency, more importantly, the collection process presumably has a negative impact on the 
patient’s credit rating. Therefore, the monetary habits of the private vendor cause a significant 
risk of substantial harm. However, because this harm is not health-related, I have not included 
the issue of patient financial burden in the discussion above. 
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Recommendation 57: 

I recommend that the Legislature’s instruction to ADC to privatize correctional health 
services. (Arizona House of Representatives, “HB 2010: Criminal Justice; Budget 
Reconciliation,” 2009) be rescinded or overridden so that ADC can return to self- 
operating health care services. 

Plaintiffs concur. Defendants take no position. 

Other Important Barriers to Compliance and Recommendations to Alleviate Them 

Re-establish “Open Clinics” 

Issue: 

Prior to May, 2017, patients accessed health care services for non-urgent episodic 
problems by submitting a Health Needs Request (HNR) form. Nurses triaged the request 
and then scheduled the patient for a visit, typically with a nurse. In May, 2017, Corizon 
changed the access mechanism by creating “Open Clinics.” Instead of submitting an 
HNR, patients were instructed to simply go to the clinic during specified hours where 
they would see a nurse on a first-come first-served basis. Based on input I received from 
both Parties, as well as wardens, custody staff 76 , health care staff, and indirectly from 
patients, the Open Clinic system improved access to health care. However, there were 
some reports of problems with access to health care. Unfortunately, the open clinic, as 
designed in 2017, had the unintended consequence of losing the ability to document any 
such access problems because the HNRs - a key data source for measuring access via 
PMs 36 and 37 - no longer existed. For this reason the previous Court ordered in June, 
2018, that patients be once again allowed to request to be seen by placing an HNR in the 
HNR box. (Doc 2901) Corizon determined that it was operationally unable to operate 
both the HNR-based and Open Clinic systems, and thus, discontinued the Open Clinic. 

It is my opinion that the Open Clinic model increased access to care and was more 
efficient, and thus that discontinuation of Open Clinics has negatively impacted the safety 
of care, both directly (e.g. PMs 36 and 37) and indirectly (other PMs, due to diversion of 
resources to an inefficient system). It is further my opinion that the Plaintiffs’ legitimate 
concerns about some cases of worsened accessibility (e.g. disabled patients who, on 
occasion, were unable to be seen, and thus had to travel to the clinic twice) and about loss 
of ability to measure such access problems (because the HNRs no longer existed), are 
solvable by a better design of the Open Clinics. Finally, if the Open Clinic model is 
designed and operated properly, there is no danger to patients if access to episodic care 
via HNRs is eliminated. 


76 There was a single complex in which custody staff were ambivalent about the advantages of 
open clinics vs. the old system. However, they also stated that they would have no objections to 
returning to the Open Clinic system. 
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Recommendation 58\ 

I recommend the Court reverse the Court’s Order (Doc. 2901) requiring that ADC allow 
patients to use HNRs to request access to health care. Along with this, ADC’s vendor will 
re-implement the Open Clinic system for all patients except for those who do not have 
freedom of movement to attend an Open Clinic, i.e. those in maximum custody, on 
mental health watch, in IPC, and in CDU who will continue to submit HNRs for their 
healthcare needs. The new Open Clinic system will address actual or potential 
weaknesses in the old Open Clinic system by incorporating the following elements: 

1. COs will maintain a list of each patient who presents to the Open Clinic for 
episodic care, whether or not the patient is seen at that time. The list will include 
the patient’s name, ADC number, and his or time of arrival. This list will serve as 
the record of patients presenting to clinic and as the Source Document for PMs 
related to the timeliness of access to episodic care. 

2. Priority for non-urgent or emergency care will always be given to disabled 
patients. 

(These patients are identified as those designated in AIMS as Special Needs 
Offenders (“SNO”). 

3. ADC will assure that there is adequate protection from the elements and seating 
for disabled patients in the waiting areas. 

4. Patients will still be able to submit HNRs, but only for non-symptom related 
needs, e.g. requesting the results of a test; requesting a medication refill. 

5. ADC will assure that patients who are programming or work offsite have 
access to Open Clinic without the need to miss programming or work. 

Defendants concur. Plaintiffs note that they “do not oppose re-starting Open Clinics in 
theory. However, we think that the HNR boxes should still be on the yard as an alternate 
way to seek care.” While at first blush, having more routes of access seems like a “good 
thing,” Corizon’s assertion that it could not practically run two systems simultaneously 
(HNRs and Open Clinics) was, in my opinion, reasonable. The current (and future 
vendors) may feel that they can run two systems simultaneously, which my 
recommendation would permit. In fact, the recommendation does envision a dual system 
(HNRs for those in restricted-movement housing), but the HNR portion of the system 
will be very small and should be manageable. 

Reduce Vacancies under Privatized Health Care 77 

Issue: 

The prior and current health care contracts allow ADC to “penalize” the vendor when the 
vendor fails to keep positions filled. In fact this is not a penalization, but rather a 
reimbursement to ADC. In other words, if the vendor does not spend $10 because a 
position was vacant, it reimburses ADC $10. Reimbursement does little to motivate the 


77 This issue and recommendation would be moot if ADC re-established self-operation. 
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vendor to keep positions filled because at the end of the transaction, the vendor is whole. 
Filling positions is critical to safe operations, and the failure to meet the required 
threshold on at least some PMs (based on my knowledge of and experience with 
correctional health care systems) is due to understaffing of budgeted positions. Further, 
even if contractual “penalties” must be limited to liquidated damages, i.e., reimbursement 
for the expected value of losses (risk) due to failure to perform, then the current 
reimbursement arrangement ignores that risk. The risk that exists from underfilled 
positions is the risk of harm to patients. 

Recommendation 59: 

ADC’s contract with the vendor should be modified such that the amount paid to ADC by 
the vendor reflects an estimation of the total cost/risk to ADC when budgeted positions 
are vacant. This must, therefore, be an amount significantly greater than the dollar 
amount of the unpaid wages. 

Plaintiffs concur. Defendants take no position. 

Redesign the Process for Fulfilling Provider Requests for Specialty Services 

Issue: 

Referrals to specialists is a key component of safe provision of health care. The 
timeliness of the processing of such referrals (from the date requested until the date the 
patient is seen by the specialist) is measured by two companion PMs: PM 50 ( Urgent 
specialty consultations and urgent specialty diagnostic services will be scheduled and 
completed within 30 calendar days of the consultation being requested by the provider.) 
and PM 51 ( Routine specialty consultations will be scheduled and completed within 60 
calendar days of the consultation being requested by the provider.) Performance on 
these two measures was quite poor. While PMs 50 and 51 suffer from the generic barriers 
to compliance described above, they suffer from two additional unique barriers: (1) the 
poor design of the Utilization Management (UM) system Corizon used for processing and 
approving (or denying) specialty requests, and (2) challenges finding community 
specialists willing to see ADC patients. The latter barrier is discussed above (see Increase 
Community Specialist Fee Payments). 

Corizon’s UM system was poorly designed. The process for handling requests for 
specialty services was as follows. Providers enter their requests for these services into 
eOMIS. A clerk at the facility transfers this request manually from eOMIS to another 
software program used by Corizon (“CARES”). The information transferred into CARES 
is limited to what the provider requests to have communicated plus whatever the clerk 
believes may be helpful to the UM decision-maker. This CARES information is reviewed 
by a UM decision-maker in Corizon’s home office. Corizon made a business decision to 
not provide UM decision-makers with access to eOMIS. Instead, if the decision-maker 
has clinical questions, the answers to which he or she needed to decide if the specialty 
request should be approved, he/she posted the questions to CARES. The facility clerk 
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then manually transferred the questions back to eOMIS, where the requesting provider 
would find them, and respond. The response followed the same pathway as the original 
request, and so on. 

The process was unnecessarily cumbersome, which contributed greatly to the observed 
delays. Two aspects of the process are particularly noteworthy. First, based on my 
review, the answers to almost all of the questions posed by the UM decision-maker are 
already in the patient’s medical record (eOMIS) and could have been easily answered if 
the decision-maker had looked in eOMIS (which is easily accessible from anywhere). 
Thus the time consumed by the back-and-forth question-and-answer was unnecessary. 

Second, one particular link in the chain - the need for a clerk to manually move data from 
eOMIS to CARES and back again - adds considerable risk for delays. To examine that 
more closely, on 2/6/19 I asked ADC staff to generate a report of all (i.e., throughout all 
10 complexes) requests for specialty services Corizon was aware of on that day where the 
request was in “Clinical Coordinator Initiated Status.” These are requests where the clerk 
who is responsible for manually moving the request from eOMIS to Corizon’s CARES 
software program has acknowledged seeing the request, but has not yet made the manual 
move. In other words, these requests are in limbo because the UM department is not yet 
aware of them and cannot make an approval/denial decision. The report, run on 2/6/19, 
showed: 

-There were 28 “Urgent” requests for specialty services which had been requested 
at least 30 days earlier. These requests should have been completed (i.e., patient 
seen by the specialist) in 30 days. Instead, at the 30-day mark or longer, they were 
in limbo, not yet having even been sent to the UM department for consideration. 

-There were 57 “Routine” requests for specialty services which had been 
requested at least 60 days earlier. These requests should have been completed 
(i.e., patient seen by the specialist) in 60 days. Instead, at the 60-day mark or 
longer, they were in limbo, not yet having even been sent to the UM department 
for consideration. 

Thus, by design, Corizon’s UM process was cumbersome, time consuming for the facility 
provider (and clerk), slow, and prone to error by miscommunication (the game of 
“telephone”). 

Recommendation 60: 

Specialty requests should be managed wholly within eOMIS. UM decision-makers 
should have access to eOMIS and use it initially to answer any questions they have about 
the request. I have discussed this with the corporate medical director for the in-coming 
health care vendor, Centurion, and have been informed that Centurion plans to manage 
UM as recommended here. 
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Defendants concur and note that they are in the process of implementing the 
recommendation. Plaintiffs concur, but note, “We also think that the 14 days timeframe 
on PM 48 is too long, and should comport with their internal policy of 3 days for a 
response by UM to urgent requests and 5 days for routine requests.” Decreasing the turn¬ 
around time for notification of denials is certainly not undesirable, but I view the delays 
in processing requests, addressed by this recommendation, and the errors in the quality of 
denials of requests, addressed in Part IV, “Utilization Management (UM) Process - Part 
1: Denials of Specialty Referral Requests” to be much more pressing issues, and issues 
which, when addressed, may naturally address Plaintiffs’ suggestion. 
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Part IV - Whether the PMs by themselves accurately reflect the adequacy of the care being 

provided (Doc. 3231 at 2) 

This part of the report addresses potentially problematic aspects of care delivered at ADC that 
are not measured by the existing 103 PMs. My method for developing this part was as follows. I 
analyzed errors in care that I encountered during my review for the first three parts of this report. 
Because that part of my review tended to center around errors in care which were measurable by 
the existing PMs, I sought another source that was not as closely tied to the PMs. The most 
relevant source I found was mortality reviews. I thus reviewed all the 58 deaths which occurred 
between September, 2018 to April, 2019 in addition to a small number of other deaths occurring 
outside that timeframe. To supplement that, I also reviewed the approximately 50 consecutive 
advocacy letters sent by the Plaintiffs to the Defendants between April and June, 2019. When I 
encountered an error, I considered whether that error would be detectable by an existing PM. If it 
would not, and the error posed a significant risk of serious harm, I addressed it here. 

The issues I address in this part of the report are based on errors I encountered once, or more 
than once, during my review. I did not conduct a systematic evaluation of the adequacy of all 
health care delivered at ADC. It is important to note that as a result, my inclusion of an issue here 
does not necessarily mean that there is a systemic problem with care at ADC relative to that 
issue. It simply means that there is an aspect of care - as evidenced by one or more actual 
examples, not just that an error which theoretically might occur - for which the PMs by 
themselves do not accurately reflect the adequacy of care being provided. 

For each issue discussed, I also suggest how ADC might monitor (and therefore measure) the 
adequacy of care related to that issue. It is worthwhile to note that most of those suggested 
measures require the monitor to use clinical judgment. Such tools are called intrinsic measures. 
These are in contrast to most of the PMs contained in the Stipulation, which do not require 
clinical judgment 78 . Such tools are called extrinsic measures. Generally, extrinsic measures 
assess whether a task was completed, or completed on time, whereas intrinsic measures assess 
whether the task was completed appropriately. Extrinsic measures are necessary but not 
sufficient by themselves to reflect the adequacy of care being provided. Intrinsic measures are 
sometimes not as easy to measure as extrinsic measures, and they also may require more effort 
(more discussion and possibly the input of a “tie-breaking” third party) to adjudicate when the 
results are challenged by the vendor. 

Quality of Clinical Decision-Making by RNs 

Issue: 

RNs are given a tremendous amount of responsibility in ADC to independently manage a 

broad spectrum of health conditions which are ordinarily managed by providers in the 

community. Such activity may be within the RN’s legal scope of practice, but is often 


78 See footnote 21. 
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beyond his or her abilities in terms of knowledge and experience. This places patients at 
significant risk of serious harm. The following are two examples: 

• Patient 36 submitted an HNR on 3/21/18 complaining that his hernia was “getting 
worse and more painful” and he was bleeding rectally. He was not seen for this 
complaint until 3/26/18. When seen, an RN observed that he had a significantly 
large hernia. She did not conduct an examination of the hernia to see if it was 
reducible (in fact, she did not touch it herself - she had the patient examine it 
himself to see if it was tender). She did not make any examination of the rectum. 
She did not refer the patient to a provider. Instead, on her own initiative she 
instructed the patient to take ibuprofen for pain, test his stool for blood on cards 
she gave him, and issued him a hernia belt. Hernias are common problems and are 
usually not serious. However, when they become incarcerated (non-reducible; 
cannot be reduced in size by pushing the contents of the hernia - the intestines - 
back into the abdomen), they present a great risk for becoming strangulated, a 
life-threatening condition. Thus it was incumbent on the nurse to examine the 
patient manually to see if the hernia was reducible. She did not. Having failed to 
do this, it was impossible to know if the hernia were reducible or not. Without 
knowing if the hernia were reducible, the nurse’s prescribing of a hernia belt was 
dangerous, as it increased the risk that the intestines would become strangulated if 
the hernia were incarcerated 79 . 

• Patient 22 is a patient with diabetes who submitted an HNR on 12/1/18 because 
two toes were swollen and painful. He was seen by an RN on 12/2/18 who 
observed that two of his toes had full-thickness wounds and the “slough does not 
allow for full staging of wound.” His blood sugar was measured at 244 (elevated). 
Other than these two things, the nurse failed to obtain vital signs or conduct a 
basic examination including examining of the toes or foot to see if they were red, 
swollen, or showed evidence of local or regional spread of infection. The nurse 
referred the patient to see a provider, but scheduled this as a “Routine” (within 14 
days) referral. The patient’s presentation suggested that he might have an 
infection, which, especially in a patient with diabetes, is an urgent problem. It 
required a much more careful evaluation looking for infection, and in the absence 
of something that would have indicated there was an emergency (e.g. elevated 
temperature, which the nurse failed to measure), required arrangements for the 
patient to be seen by a provider later that day or the next day (i.e., “Urgent” 
referral). Instead, the nurse did not arrange for a provider visit until two days 
later, on 12/4/18. This delay could have made the difference between a treatable 


79 When the patient presented again with continuing pain in December of 2018 and was referred 
to a surgeon, the surgeon found that his hernia had in fact become incarcerated and 
recommended surgery “ASAP.” 
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infection and a more serious infection with possible amputation. Thus the nurse 
placed the patient at significant risk of serious harm. 80 - 81 

An aspect of health care that cannot be fully disentangled from the actual decision¬ 
making is the documentation of that decision-making. While this is a generic issue for all 
health care disciplines, there is a specific situation in the nursing realm that requires 
specific attention. ADC relies heavily on “documentation by exception” (DBE). DBE is a 
practice by which nurses evaluating a patient use a preprinted form which lists various 
possible positive findings related to the issue at hand. If the patient shows evidence of 
one of the positive findings, the nurse checks that finding on the form. In the absence of a 
mark (the exception), the reader is supposed to conclude that the finding was not present. 
This practice is used to save time and is an acceptable form of documentation. However, 
it is practiced improperly at ADC. For example a nurse evaluated Patient 56 on 2/16/19 
for a change in behavior, documenting on a “Suicide Watch Progress Note-Follow up - 
Objective” form. The nurse filled out parts of the form, but left other parts empty. So it is 
impossible for a subsequent care provider who needs to rely upon the record, to know 
whether the empty parts reflect that the patient did not demonstrate the finding, or the 
nurse just did not conduct that part of the examination. The scope of PM 5 ( Medical 
Records will be accurate, chronologically maintained, and scanned or filed in the 
patient’s chart within two business days, with all documents filed in their designated 
location.) is limited to the accuracy of filing of scanned HNRs; there is no PM that 
examines whether clinical notes by nurses are complete and comprehensible. 

Candidate Metric: 

Care (and the documentation supporting that care) delivered by RNs is clinically 
appropriate. 

Quality of Clinical Decision-Making by Medical Providers 

Issue: 

Medical providers make clinical decisions in three general settings: during face-to-face 
patient encounters; in response to an inquiry from a nurse; or upon receipt of results from 


80 The patient was transferred to Maricopa County Jail before the provider visit took place. There 
is no evidence that the nurse, provider, or any other Corizon professional took any steps to notify 
the receiving facility of the patient’s condition or urgent needs. 

81 In their comments to the draft of this report, addressing both examples, Defendants noted, 
“Medical discretion. Dr. Stem admits that these are within an RN’s scope, but beyond 
experience and knowledge. Practitioners often encounter things outside of their experience and 
have medical discretion to learn or refer out. In both cases, the RN referred to a provider. Again, 
this is difference in medical opinion.” Defendants are correct that the actions described were 
within the RNs’ legal scope of practice. However, practicing within scope is necessary, but not 
sufficient for patient care to be safe. Further, medical discretion describes the choice between 
two reasonable courses of action. The care I have described illustrates a different issue: the 
choice between a reasonable and an unreasonable (and in this case, dangerous) course of action. 
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a test, a report from a consultant, or other external medical record. Some PMs measure 
whether, and how timely, providers conduct some of these activities, but none measure 
the quality of the care the provider delivers during the activity. I found many examples of 
poor quality clinical decisions made by medical providers in these three settings; most of 
these were made by mid-level providers. 82 The following are examples: 

• Patient 27 was a 64-year old male with a history of hepatitis C, hypothyroidism, 
high blood pressure, high cholesterol, and obesity, who was seen by an RN on 
1/31/19 for complaints of lightheadedness, shortness of breath, abdominal pain, 
dry mouth, nausea, and lethargy. The nurse discussed the case with an NP who 
advised her to give the patient a medication for nausea and stomach acid, order 
two blood tests (Diagnostic Panel 2 and Complete Blood Count). The NP’s 
clinical decision was inadequate in that stomach problems would be a very 
unlikely cause of all the patient’s symptoms. Based on his symptoms - and 
especially in light of his risk factors - the patient might have been suffering from 
other more serious conditions. The blood tests might help diagnose one of those 
other conditions; however, to do so safely required obtaining the tests in the next 
few hours or days. Instead, based on the NP’s instructions, they were not done 
until 2/12/19. On 2/13/19 the results of those blood tests were reported back to the 
facility. They were quite abnormal, including a very elevated white blood cell 
count (22,000, normal 6,000 -10,000) which is usually a sign of serious active 
infection or other severe inflammation, and is an “alarm” result until proven 
otherwise. The above-cited risk was still present, but more obvious. However, a 
physician reviewed them on 2/15/19 and took no action other than indicating that 
the result should be reviewed at the patient’s next chronic care clinic visit. Not 
only was immediate action required, but even if review at the next chronic care 
visit were appropriate, no such visit was on the schedule nor did the physician 
schedule it. On or around 2/21/19 the patient complained again about abdominal 
pain leading to admission to a community hospital, where his abdominal pain was 
diagnosed as metastatic pancreatic cancer, from which he died in mid-March. 83 

• Patient 21 was a 41 year old male. He was admitted to ADC on 5/9/17 at 18:00. 
He reported no history of medical problems but was on anticoagulants (blood 
thinners). An hour after admission medical staff responded to an emergency 
called because he was suddenly behaving abnormally. His heart was racing (136), 
and was found to have sweating, piloerection (“goose bumps”) and dilated pupils. 


82 Given my methodology, I cannot opine on whether the frequency of errors among mid-level 
providers relative to physicians is due to a real difference in quality of care between the two 
groups of providers, or the fact that a much larger proportion of all care is delivered by mid-level 
providers. 

83 I am omitting the exact date to ensure patient confidentiality because dates of death are 
searchable in the public record. 
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The nurse was unable to obtain other vital signs. The patient was seen by an NP 
who documented: “Called to assess IM for what appears to be seizure activity. 
Upon arrival in Medical, IM in a prone position on a stretcher in ankle and wrists 
restraints, yelling with purposeful movements, attempting to lift self up on 
stretcher continuously, combative, uncooperative to verbal commands, spitting at 
staff, spit mask applied by DOC officers. [Alert and oriented to person, place, 
time], using profane language. No seizure activity witnessed, appears to be 
withdrawing from unknown substance. Unable to perform physical exam, visual 
observation only. Diaphoresis, Blood sugar checked: 111, denies [diabetes] or 
Psych history. Denies polysubstance abuse; Meth, Heroin, PCP, Spice, or Bath 
salt. Report "I only smoked cigarettes from QT". Denies chest pain, chest 
tightness, chest pressure, abdominal pain, back pain, or any pain at all.” Her 
diagnosis was “Ingestion of unknown substance/withdrawal symptoms?” The NP 
ordered the emergency antidote for opiates (naloxone 2 mg.) and “cleared from 
Medical” to be placed in dry cell by custody, with “follow up Provider or Nursing 
line PRN.” The NP’s decision-making was illogical and dangerous for a number 
of reasons, including: (1) There was little if anything about the patient’s 
presentation to suggest intoxication with opiates, thus administration of naloxone 
made no sense. (2) A sudden change in a patient’s physical and mental health 
condition, especially if thought to be due to a foreign substance, is a critical time 
for a patient and requires close medical and mental health (because of a risk of 
suicide) observation; not release back to a non-medically monitored environment. 
Some of the very possible non-substance-related diagnoses from which the patient 
might have been suffering at the time (such as excited delirium and encephalitis 
(infection of the brain)) required emergency treatment and evacuation to the 
hospital. (3) The presence of anticoagulants in the patient’s blood put him at risk 
for potentially fatal internal bleeding as a result of even relatively mild trauma 
(such as might have occurred during the struggle with COs or from banging his 
head), again dictating the need for careful medical monitoring. 

The following morning a physician assistant “Arrived to do [physical 
examination], yet inmate was uncooperative/aggressive; was not able to be 
assessed at that time. Was banging his head on the wall and jumping/flailing his 
arms as well. CO was trying to talk him down with no avail. Due to his erratic 
behavior, I exited the area; no [physical examination] could be done at that time. 
Upon returning a short time later, inmate was on floor with nurse over him; he 
had hit his head on the toilet and somehow fell backward hitting his head yet 
again. Inmate nonresponsive.” For all the same reasons described above, this 
patient’s condition demanded the provider’s intensive medical evaluation and 
care, not his departure, and therefore the decisions he made created a significant 
risk of serious harm. At 09:30 medical staff again responded to an emergency 
called by COs because “Medical requested to stand by as I/M was restrained by 
security. I/M yelling, thrashing around on floor. 2 officers were restraining upper 
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body. Behavior continued for a few minutes until officer stated I/M had no pulse. 
FHA Watts entered cell palpated & reported thready & weak pulse to carotid.” 
Resuscitation efforts were started. He died shortly after arrival in the hospital. The 
cause of death was cardiac arrest in the setting of methamphetamine toxicity, 
hypertensive and atherosclerotic cardiovascular disease, and physical exertion 
while being restrained in the prone position by Security staff.” Appropriate 
decision-making by either provider had a high likelihood of preventing the 
patient’s death. 

Candidate Metric: 

Care (and the documentation supporting that care) delivered by providers during a face- 
to-face patient encounter, in response to an inquiry from a nurse, or upon receipt of 
results from a test, a report from a consultant, or other external medical record, is 
clinically appropriate. 

Chronic Disease Management - Medical 

Issue: 

In Section I of this report, I discuss the technical weaknesses in how PM 54 ( Chronic 
disease inmates will be seen by the provider as specified in the inmate's treatment plan, 
no less than every 180 days unless the provider documents a reason why a longer time 
frame can be in place.) and PM 55 ( Disease management guidelines will be implemented 
for chronic diseases.) are currently being measured and make recommendations for 
improvements in their measurement. However, even with those recommended changes, 
these PMs are still insufficient to measure safe management of chronic diseases for two 
reasons. First, they fail to measure the adequacy of care for chronic conditions beyond the 
20 cited in the reference list. Second, they are limited to measuring adherence to return 
visit time intervals and some testing; the adequacy of provision of chronic care requires 
examining the quality of the care delivered, not just its timeliness. The following 
examples illustrate both points. 

• Patient 40 has a history of treated prostate cancer for which a blood test 
(PSA) was being measured periodically to monitor for recurrence of the 
cancer. In early 2018 his PSA was noted to be elevated, a sign of possible 
recurrence. It is not clear to me how long that had been going on for. His 
care providers suspected the elevation might be due to a prostate infection, 
not cancer, so they treated him with antibiotics. But despite that treatment, 
the PSA remained high, reinforcing the likelihood that cancer, not 
infection, was the cause. By 2/28/18, the providers finally concluded that 
he needed to be referred to a urologist, and therefore submitted their first 
request for specialty consultation on that day. Over the next year, there 
were numerous delays, including delays introduced by the Utilization 
Management process wherein the UM department essentially 
recommended that the providers at the complex - providers who are 
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generalists, not urology or cancer specialists - manage the cancer issue 
themselves, as reflected in their “NMI” response (response to the specialty 
request indicating that the UM department needs more information) to the 
2/28/18 referral request: 

Does the DRE [digital rectal examination] confirm prostatectomy? 
If cannot find path[ology report of previous cancer] can access the 
Arizona state tumor registry (if the surgery was in Arizona) 
cancer is a reportable disease and the state registry can help us with 
this 

do not know what urology has to offer 
if dre+ for prostate - needs repeat biopsy 
if dre - for prostate- he has advanced disease, and can consider 
bone scan and it [sic] positive consider on site hormones 
if local recurrence (by dre) can consider xrt [radiation] 

PLEASE SUBMIT DRE EXAMINATION 

Over the next several months the UM department vacillates between 
recommending that the facility providers switch their specialty request 
from Urology to Oncology or Oncology to Urology. Further delay is also 
introduced by requesting providers to get old records and acceptance of 
Alternative Treatment Plans recommended by the UM department. 
Ultimately, the patient was seen by a urologist on 4/17/19. The urologist 
recommended the patient be referred to an oncologist for management of 
prostate cancer. The patient was seen by an oncologist on 7/8/19, now 
nearly a year and a half after concern was raised about recurrence of 
cancer. If the cancer recurred, depending on the extent and location of the 
cancer, it is potentially curable. However, delays in diagnosis and 
treatment reduce the chances of curability. Thus the delays in managing 
this patient’s chronic condition posed (and continue to pose 84 ) a significant 
risk of serious harm. 

Candidate Metric: 

Care provided (and the documentation supporting that care) for all chronic medical 
conditions is clinically appropriate. 


84 As of 8/9/19, the bone scan recommended by the urologist on 4/17/19 and again by the 
oncologist on 7/8/19 has not been requested, and the CT scan recommended by the oncologist on 
7/8/19 has been requested, but not yet performed. These errors are already measured by PM 52 
{Specialty consultation reports will be reviewed and acted on by a Provider within seven 
calendar days of receiving the report.). 
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Mental Health Treatment Plans 

Issue: 

In the mental health arena, treatment plans are the road maps to future care. They 
describe what the patient’s problem(s) or need(s) is (are), which one(s) is (are) going to 
be addressed in the next time period, the goal in addressing the problem, how the goal is 
going to be met, and how the clinician and patient will measure the efficacy of treatment. 
PM 77 ( Mental health treatment plans shall be updated a minimum of every 90 days for 
MH-3A, MH-4, and MH-5 prisoners, and a minimum of every 12 months for all other 
MH-3 prisoners.) measures whether or not the clinician has filed a treatment plan. 
However, it does not measure the adequacy of that treatment plan. 

The review I conducted with Dr. Abplanalp’s assistance demonstrated that treatment 
plans filed for ADC patients are generally inadequate. The goals were often vague or 
generic with few if any action steps, treatment strategies, or criteria of efficacy identified. 
The absence of an adequate treatment plan does not automatically equate with 
inadequate care, though it may in some cases. At the very least, in the absence of a 
clearly articulated treatment plan, it is difficult for other concurrent (or subsequent) care 
providers to know what is (was) going on with the patient and how to integrate their 
efforts with those of the primary clinician. 

Candidate Metric: 

All patients on the mental health case load have documented clinically appropriate 
treatment plans with the following elements (when appropriate): 

• Problems/Needs : These are based on the impact of the patient’s symptoms on his 
or her subjective distress and functional impairment, and are not simply a 
reiteration of the diagnosis. Thus one patient with depression may have a need to 
increase his or her level of motivation whereas another with the same diagnosis 
may have a greater need to address chronic suicidal ideation. The treatment plan 
addresses the highest priority need(s), not necessarily all needs. It also draws from 
and builds upon the outcomes for the needs and related goals (see next bullet) set 
forth in the previous treatment plan. In other words, if a previously identified 
priority need has been successfully addressed, it may be appropriate to identify a 
new problem. 

• Goals : These are not just stated as the absence of the identified problem, but in 
terms of outcomes that are measurable, i.e. how, specifically, the patient will be 
functioning better if the problem is successfully addressed. The goal, when 
couched this way, also serves as the ruler by which to measure whether the 
intervention was successful. 

• Action Steps/Interventions : These are the steps the patient and or clinicians will 
be taking to address the identified needs. They are specific strategies or actions 
that are tailored to the patient and their specific constellation of issues. As with 
the first bullet, these steps draw upon the outcomes of the previous treatment plan. 
An unsuccessful intervention drafted in the previous treatment plan may require 
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intensification, modification, or replacement. As part of designing interventions, 
the treatment plan includes helping the patient recognize qualities they have that 
can assist them in achieving their goals and characteristics or patterns that get in 
the way. 

Management of Suicidal Patients on Watch 

Issue: 

Existing PMs assess the frequency with which patients placed on suicide watches are 
monitored and by whom they are removed from watch. However, and without regard to 
the Court’s decision on Recommendation 16 (whether short visits satisfy the requirement 
of several PMs that patients be “seen”), existing PMs do not measure the adequacy of 
actual care delivered during the watch. Based on my review with assistance from Dr. 
Abplanalp, the content of care delivered by mental health clinicians to patients on mental 
health watch is not always adequate. Whether limited by time, skill, policy, or other 
environmental factors, patients did not always have adequate assessments of their level of 
suicidal risk or attention paid to whether factors contributing to their suicidal ideation 
exist. Two important factors contribute to inadequate assessments. The first is the 
frequency with which assessments are conducted in a non-confidential setting. The 
Stipulation requires that the mental health clinician conduct encounters in a confidential 
setting unless the patient refuses. ADC is highly compliant with this requirement as it 
applies to encounters during watch (PM 94) in that when the encounter is not conducted 
in a confidential setting, there is adequate documentation that the clinician offered and 
the patient refused. However, conducting these encounters in a confidential space is of 
paramount importance for patients on watch because it helps ensure that the patients 
share complete and accurate information with the clinician, information which is key to 
assessing risk. Unfortunately, a very high percentage of the watch-related encounters I 
reviewed were conducted at the cell-front (i.e. non-confidentially). The second important 
contributing factor is the absence of a formal risk assessment tool or process. 

Inadequate assessments can result in one or more of the following errors: (1) 
inappropriate initial assignment to a particular level of watch (i.e., constant observation, 
10-minute checks, 30-minute checks); (2) inappropriate promotion to a less intense level 
of watch; (3) failure to provide adequate treatment or resolution of factors which 
contributed to the need to be placed in watch. None of these errors are accurately 
reflected in existing PMs. All three errors are illustrated in the following example. 

• On 3/25/19 at 06:46 Patient 41 was released from a suicide watch. The 
assessment conducted leading to the release was conducted at cell-front, 
i.e. in a non-confidential setting. At 13:54 that same day, an emergency 
response was initiated after she made statements of self-harm. She told the 
responding nurse that she wanted to kill herself by "strangling myself with 
a sheet or like my hands or anything I can get my hands on” and told the 
mental health clinician "I’m just tired of being bullied.” The clinician 
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wrote “she is tired of being reminded of her crime when she is here ‘trying 
to move passed [sic] it’.... IM stated that she is called a ‘Cho Mo 
everyday’. IM stated that she regrets what she did and stated that she deals 
with depression all the time. IM reported ‘I just don't want to be here 
anymore and that's on an everyday basis’. IM reported that she had a plan 
to hang herself with a sheet.” She was placed back on watch, but despite 
the fact that she had suicidal ideation and a plan - placing her at high risk 
for suicide and therefore in need of constant observation - she was placed 
on 10-minute observation (the next less intense level of watch). A mental 
health clinician evaluated her every day thereafter. However, each of these 
evaluations - on 3/27, 3/28, 3/29, 3/30, 3/31, and 4/1 - were conducted at 
cell-front, each following an offer, and refusal, of evaluation in a 
confidential setting. Based on the last evaluation, and without adequate 
assessment and addressing of factors that led to her original placement on 
watch, she was released from watch on 4/1/19 at 07:22. Not surprisingly, 
she once again expressed thoughts of self-harm later that day necessitating 
her placement back on watch in the evening of 4/1/19. The assessment for 
this re-placement on watch was conducted by an RN. The RN elicited 
from the patient that she “would use my shoelaces to strangle myself’ and 
documented that the patient had “thoughts,” “means,” and a “plan” for 
suicide. Based on the information available, there is little evidence 
indicating that this patient would not be at high risk of a suicide attempt 
and therefore required her to be placed on constant observation. However, 
based on a phone conversation between the nurse and a mental health 
clinician, the patient was placed instead on 10-minute watch. On 4/3/19, 
the patient told the clinician conducting her daily assessment that she 
would be willing to go back on her anti-depression medication 
(citalopram). Anti-depressants do not reduce depression symptoms 
immediately - this can take several days to a few weeks - and the longer it 
takes to start the medication, the longer the patient remains at risk due to 
depressive symptoms. Thus this was information the clinician should have 
communicated straight away to a provider. No such communication was 
made, quickly or at all. Instead, on 4/8/19 - while still on watch, and still 
receiving daily evaluations by a mental health clinician - the patient sent 
an HNR asking to be placed back on her medication. (She was seen by a 
provider the next day and received her first dose of the medication on 
4/11/19). The inadequate care this patient received over the course of two 
back-to-back suicide watches (failure to adequately assess her continuing 
risk prior to release from the first watch; failure to place her on constant 
watch at the initiation of the first and second watch; and failure to inform a 
provider of the patient’s willingness to restart an anti-depressant thus 
delaying treatment) placed her at a significant risk of serious harm. 
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Candidate Measuring Tools: 

1. Mental health care (and the documentation supporting that care) provided during 
suicide watches, including, but not limited to: (1) decisions assigning the patient to a 
particular level of watch (i.e., constant observation, 10-minute checks, 30-minute 
checks), either initially, or during the course of the watch, (2) assessments of risk (static 
and dynamic contributing factors as well as protective factors) with and without the use 
of a formal suicide risk assessment, as appropriate, and (3) decisions to discontinue 
watch, is clinically appropriate. 

2. Few mental health encounters during suicide watch are conducted in a non-confidential 
setting. 

This could be quantified in a number of ways. One example would be to set a goal of 
decreasing the number of patient refusals to be seen confidentially by a certain 
percentage every month, e.g. 5%, until the refusal rate reaches a lower rate, e.g. 10%. It is 
important to note, as stated earlier in this report (Part I, “Were Mental Health Patients 
“Seen,” paragraph I) that some of the cell-front watch-related encounters conducted by 
mental health clinicians are appropriate. Thus the target of a measure like this one should 
not be 0%. 

Success on this measure would require a broad operational and cultural change, to 
include changing the expectations of clinicians, COs, and even patients. For example, the 
policy of shackling patients when taking them from their cells to private rooms to meet 
with the mental health clinician merits scrutiny. Currently patients on watch are housed in 
living units designated as high level of custody. Many, if not most, of these patients do 
not meet the criteria of high custody. However, they are still subjected to the 
requirements of high custody (notably shackling before removal from the cell). It is likely 
that the prospect of having to be shackled serves as a deterrent to agreeing to be taken out 
of their cell. It is also possible that CO and mental health clinician staffing levels would 
need to be adjusted because transferring a patient from his or her watch cell to a 
confidential setting is more time-consuming than cell-front encounters, not only because 
the transfer takes time, but also because the encounters are likely to last longer. 

Management of Mental Health Patients, Generally 

Issue: 

The previous section discusses the clinical management of patients on suicide watch. 

This is a subset of the broader topic of management of mental health patients. As with 
management during watch, there are PMs that measure whether certain clinical activities 
are done and whether they are done timely, but no PMs that measure the adequacy of the 
care delivered. And, as with management of patients on watch, the content of care 
delivered by mental health clinicians to patients in non-watch settings is also not always 
adequate as illustrated by the following case: 
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Patient 2 was released from suicide watch on 3/21/19. (It should be noted 
that this watch was the last in a series of five back-to-back placements on 
suicide watch in the previous four weeks, with gaps between release from, 
and replacement on watch ranging from three hours to one day as shown 
below. 


Watch began 


Watch ended 


2/23/19 


2/25/19 (10:49) 
3/4/19 


2/25/19(13:17) 


3/5/19 

3/15/19 


3/14/19 

3/18/19(06:35) 


3/18/19(14:06) 3/21/19 

See discussion above in “Management of Suicidal Patients on Watch” 
above.) She was seen for her first post-release encounter on 3/22/19 
(conducted cell-front) and her second post-release encounter on 3/28/19 
(conducted in a confidential setting). During neither encounter did the 
clinician explore the specific underlying reasons for her frequent severe 
episodes requiring placement on watch. Therefore she remained at 
significant risk of these episodes recurring. 

Candidate Metric: 

Mental health care (and the documentation supporting that care) provided during 
non-suicide watch encounters is clinically appropriate. 


Treatment of Substance Use Disorder 

Issue: 

During the months of August to December 2018 alone, there were seven deaths at ADC 
due to drug intoxications (six were due to opiates alone; in the seventh, “spice” may have 
been the main drug, but heroin was also found in the patient’s blood). Of these, two 
patients (Patient 44 and Patient 13) had had previous acute intoxications during their 
incarcerations. According to his Mortality Review, Patient 13 received “drug counseling” 
after his first intoxication, but not after his second. Patient 44 received no treatment for 
substance use disorder (SUD). 85 


85 In their response to a draft of this report, Plaintiffs note, “ADC also has changed its policies 
such that anybody who is treated for an overdose is charged for all medical care, transportation 
to outside hospitals, etc. Plaintiffs believe that this policy is counterproductive and dangerous, 
and will result in fewer people seeking treatment for their substance use disorders. See 
https://theappeal.org/prisoners-in-arizona-now-charged-for-their-own-drug-related-hospital- 
visits/ and DO 803, section 8.3.1 at 

https://corrections.az.gov/sites/default/files/policies/800/0803_032519.pdf (page 14 of 
document)” I did not independently confirm ADC’s policy. However, I do not believe patients 
choose to overdose, thus I do not believe that the such a policy would affect patients’ willingness 
to participate in a post-overdose treatment program. 
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SUD is a serious medical condition with sometimes fatal outcomes, as seen in these 
seven cases. Many correctional systems defer SUD treatment until months prior to 
discharge under the presumption that patients are safe from the dangers of SUD while 
incarcerated because they lack access to illicit drugs. However, once a patient has 
demonstrated that he or she can, indeed, access illicit drugs, that presumption is no longer 
valid. At that point, then, deferral of SUD treatment is also no longer reasonable. Thus 
these two patients should have been offered appropriate drug treatment. There is no PM 
that assesses whether patients who require SUD treatment are offered or provided such 
treatment. 

Candidate Metric: 

Patients with SUD who demonstrate continued use of drugs of abuse are offered 
treatment that is disease-appropriate and patient-appropriate 86 at the time the continued 
use is discovered, whether or not release from prison is imminent. 

Management of Patients During an Emergency Response 

Issue: 

PM 25 {A first responder trained in Basic Life Support responds and adequately provides 
care within three minutes of an emergency.) addresses the adequacy of initial care 
provided by the first responder (typically a CO) to an emergency. There are other 
components to an emergency response including the subsequent care provided by 
medical staff, joint care provided by medical and custody staff, coordination of the care 
with community resources, and, as with all other health care, documentation of the event. 
During my review I encountered cases in which these other components were not 
adequate. As discussed in Part I of this report, the current version of PM 25 is 
problematic. However, even if the recommendations I make there (Recommendation 18) 
were implemented, neither PM 25 nor any other PM would accurately reflect the 
adequacy of these other components. The following is an example of an inadequate 
emergency response. 

• Patient 19 was involved in an altercation on December, 2019 at 20:47. 87 
An emergency response followed. He was found to have a “deep cut to 
forehead approx. 2 inches” and swelling of his wrist. His vital signs were 
markedly abnormal (blood pressure 168/101, elevated; heart rate 127, very 
elevated). The patient thus had a head injury from significant a 
significantly strong blow, requiring that his neck be kept still until he 
could be evaluated for a possible fractured neck, and requiring evaluation 


86 By “patient-appropriate” I mean that a one-size-fits-all approach to treatment would not be 
appropriate. For example, an opiate dependence program should not offer patients naltrexone 
(Vivitrol®) as the only medication option. 

87 I am omitting the exact date to ensure patient confidentiality because dates of death are 
searchable in the public record. 
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of his pupils. Further, he required evaluation of his wrist for possible 
fracture (or immobilization of his wrist until such an evaluation could be 
done). His head laceration required irrigation to clean it, and given that it 
was deep, barring some explanation to the contrary, required suturing (or a 
plan to suture it in the next few hours). Finally, his vital signs were 
unstable and required close monitoring over the following minutes to 
assure that they normalized or were addressed. With input from the on-call 
NP, the care the patient received included none of these elements and he 
was sent back to his living unit with a fresh dressing on his laceration 
which he was told to “[follow up] with medical” at some unspecified date 
and time. Further, the nurse had the patient intentionally bend his neck in 
all directions, something which, if the patient had had a fractured neck, 
had a high likelihood of causing damage to, or severing, his spinal cord. In 
sum, the acts and omissions of the nurse and NP created a significant risk 
of serious harm. 

About an hour later, at 21:54, another emergency was called due to a 
change in the patient’s behavior: he became argumentative with COs, 
yelling nonsense phrases and single words, and died later. According to 
the Mortality Review conducted by ADC, the “[Patient] obviously became 
apneic [stopped breathing] and received CPR after significant delay” 
seemingly due to a lack of coordination and/or cooperation between 
custody and medical staff, though the details are not clear. The clinical 
documentation by the medical staff was very incomplete, contributing to 
the lack of clarity of what may have happened between custody and 
medical staff. 

Candidate Metric: 

Care (and the documentation supporting that care) provided during an emergency is 
clinically appropriate. 

Management of Patients upon Admission to an IPC (Inpatient Component; Infirmary) 

Issue: 

PM 64 (In an IPC, a Medical Provider evaluation and plan will occur within the next 
business day after admission .) and PM 65 (In an IPC, a written history and physical 
examination will be completed by a medical provider within 72 hours of admission.) 
address the timing of the provider’s formal admission of a patient to the IPC. Nurses can, 
and do, admit patients to the IPC independently, and no contact with a provider is 
required until “the next business day” which, on a holiday, could be as long as three days 
later. As discussed briefly in Part IA of this report, compliance with PM 64 and PM 65 
does still not provide ample patient protection. The patients placed in the IPC are, by 
definition, the most acutely ill residents in the prison. For a patient to remain in an acute 
bed for this long without provider involvement is dangerous. No PM addresses the need 
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for a provider’s initial and immediate involvement in the admission process to instruct 
nurses on necessary monitoring and/or treatment. In Part II of this report, I used the case 
of Patient 52 as an example of how failure to evaluate the patient upon admission to the 
IPC within the one-day time requirement of PM 64 posed a risk to the patient. That same 
example applies here, in that even performing the evaluation within one day would not 
have eliminated the risks I described. 

Candidate Metric: 

A provider is contacted and collaborates on the immediate care plan as soon as a patient 
is admitted to the IPC. 

Management of Patients in the IPC 

Issue: 

PM 64 and PM 65 (above), and PM 66 (In an IPC, a Medical Provider encounters will 
occur at a minimum every 72 hours.) address the timing of provider tasks associated with 
admitting and monitoring an IPC patient. Safe patient care requires not only that tasks be 
done on time, but also that they be done competently. No PM currently assesses the 
adequacy of medical decision making by providers while patients are in the IPC. 

As illustrated in the following example, provider decision making in the IPC may not be 
competent. 


• Patient 7 was admitted to the IPC by a nurse on 5/16/19 for progressive 
weakness and vomiting. The patient had been seen on 5/3/19 by another 
provider who had concerns that the patient might have a compression of 
his spinal cord, and was awaiting approval of an MRI. He suffered a fall 
on 5/15/19 due to worsening weakness and experienced nausea and 
vomiting on 5/15/19 and 5/16/19, leading a nurse to finally admit him to 
the IPC because of his clinical deterioration. He was seen by a provider on 
5/17/19 who confirmed that the patient had decreased muscle strength in 
his right arm and leg, but despite this and the patient’s recent history, 
discharged him back to his living unit. The patient continued to deteriorate 
to the point where on 5/25/19 he was finally sent to the hospital where he 
was found to have serious brain and spinal cord abnormalities and 
underwent neurosurgical treatment. The patient’s condition on 5/17/19 
included red flags suggesting that the patient might be suffering from a 
serious medical condition, such as pressure on his spinal cord or brain, and 
required additional examination by the provider. Most importantly, it 
required continued nursing support and close observation in the IPC. 

Delay in making the correct diagnosis could have resulted in permanent 
loss of function or death. 
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Candidate Metric: 

Care (and the documentation supporting that care) provided in the IPC is clinically 
appropriate. 

Utilization Management (UM) Process - Part 1: Denials of Specialty Referral Requests 

Issue: 

UM is the process used by the vendor to vet requests by providers for community-based 
specialty services. It was described in more detail earlier in this report (Part I, 

“Processing of Requests for Specialty Services). Several PMs measure various aspects of 
the UM process. PM 50 ( Urgent specialty consultations and urgent specialty diagnostic 
services will be scheduled and completed within 30 calendar days of the consultation 
being requested by the provider .); and PM 51 {Routine specialty consultations will be 
scheduled and completed within 60 calendar days of the consultation being requested by 
the provider.) measure the timeliness of UM decisions in response to provider requests to 
refer patients for specialty consultations and diagnostic services. PM 48 ( Documentation, 
including the reason(s) for the denial 88 , of Utilization Management denials of requests 
for specialty services will be sent to the requesting Provider in writing within fourteen 
calendar days, and placed in the patient's medical record.) measures the timeliness with 
which providers are notified when one of their requests is denied. PM 49 {Patients for 
whom a provider's request for specialty services is denied are told of the denial by a 
Medical Provider at the patient’s next scheduled appointment, no more than 30 days 
after the denial, and the Provider documents in the patient's medical record the 
Provider’s follow-up to the denial.) measures the timeliness with which the patient is 
notified of the denial. While the timeliness of the steps in the decision/reporting process 
is an important dimension of care, the appropriateness of the underlying approval/denial 
decision is even more critical to patient safety and is not currently measured by any of the 
PMs. Almost all of the denial decisions I reviewed recommended less aggressive (and 
less costly) management of the patient than what the patient’s provider requested. Often, 
in my opinion, the denial or less aggressive Alternative Treatment Plan (ATP) was not 
appropriate, given the clinical information in the case, and therefore presented a 
significant risk of harm to the patient as illustrated in the following example. 

• The case of Patient 5 was already cited earlier in this report (Part n, PM 
48) as an example of how delays in denying consult requests poses a risk. 

I cite the example here because delayed or not, the denial posed a 
significant risk of serious harm. The patient was seen by a provider on 
10/29/18 for right testicular pain (and other symptoms). On examination 
he had a “firm hard nodule” on his right testicle. The provider requested 
an ultrasound of the testicle. The request was denied by Corizon’s UM 


88 Denials are communicated in one of two ways. Sometimes the UM department simply denies 
the request. Sometimes it recommends an Alternative Treatment Plan (ATP). In the context of 
the Stipulation, both are considered denials. 
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department, recommending instead “obtaining detailed history, ROS 
[taking a history of other related parts of the body], initial diagnostics ( 
cbc, crp and UA) [complete blood count, C-reactive protein - a test for 
inflammation, and a urinalysis] to determine necessity for further 
imaging.” A new hard testicular nodule is cancer of the testicle until 
proven otherwise, and an ultrasound is the diagnostic test of choice. 
Further, none of the alternatives recommended by the UM department 
would reasonably be expected to shed further light on the issue or obviate 
the need for an ultrasound. The requesting provider (a mid-level provider) 
accepted this unacceptable denial without appeal. As of early August, 
2019, an ultrasound has still not yet been performed. 89 

Candidate Metric: 

UM denials (including ATPs) are clinically appropriate. 

Utilization Management (UM) Process - Part 2: Managing Patients after Denials of Specialty 

Referral Requests 

Issue: 

Following submission of a request for specialty care, it was very common for the Corizon 
UM Department to either deny the request or recommend an ATP. The danger associated 
with this UM Department habit is discussed in the section above. However, there are two 
subsequent failsafe mechanisms which should mitigate - at least to some extent - the 
danger. The first is that the requesting provider can reject or appeal the denial/ATP. The 
provider is ultimately responsible for his or her patient, and so only the provider can 
rescind his or her original order and replace it with the ATP. The second is that once 
accepting an ATP, the requesting provider executes that ATP. I found problems with both 
of these failsafes at ADC. As with errors in Denials of Specialty Referral Requests, 
above, there is currently no PM that measures the appropriateness of acceptance of 
denials by requesting providers or their follow through with the recommendations stated 
in ATPs. 

With regard to the first failsafe - rejecting or appealing the denial/ATP - use of this 
failsafe is extremely rare. The requesting providers almost always accept the denial/ATP 
without question. An example of this is found in the case of Patient 5 presented in the 
previous section (“Utilization Management (UM) Process - Part 1: Denials of Specialty 
Referral Requests”). 


89 On 7/10/19 a provider reported a genital examination: “Normal lae [sic; male?] genitalia, no 
rash, lesion, penile discharge, erythema or scrotal swelling,” however, I cannot tell with 
certainty if the testes were examined. If they were, and there was no nodule, no current danger 
exists. However, my analysis above, which addresses the decisions made based on the 
information available at the time and without the benefit of knowledge of subsequent events, 
remains unchanged. 
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With regard to the second failsafe - the provider executing the recommendations of the 
ATP which he or she accepted -1 found that the providers often delayed or failed to use 
this failsafe as well. In the case above of Patient 5, the provider did not obtain a complete 
blood count until more than three months later, and, as of early August, 2019, has never 
obtained the C-reactive protein. The following is another example: 

• Patient 54 was seen on 11/2/18 for a 4 cm x 3 cm mass on his neck which 
had been growing and was starting to cause pain. The provider requested a 
referral to a surgeon. The UM Department responded that if the lesion is 
consistent with a benign fatty tumor (lipoma), surgery wasn’t necessary, 
but if the provider had concern that the mass was not benign, that he/she 
should consider getting a biopsy. On 11/15/18 a provider accepted the 
ATP, writing “Follow up next provider visit day.” As of early June, 2019, 
no such follow-up ever took place. 90 

Candidate Measuring Tools: 

1. Providers appeal inappropriate denials (including ATPs) of their requests for specialty 
services. 

2. When providers appropriately accept ATPs, they execute all the elements of the 
accepted ATP, and do so in a timely manner. 

Medication Provision 

Issue: 

One of the most important dimensions of safe delivery of health care is the seamless 
provision of medications. Current PMs only address this to a limited degree. PM 11 
(Newly prescribed provider-ordered formulary medications will be provided to the 
inmate within 2 business days after prescribed, or on the same day, if prescribed STAT.), 
PM 13 ( Chronic care and psychotropic medication renewals will be completed in a 
manner such that there is no interruption or lapse in medication.), and PM 14 ( Any refill 
for a chronic care or psychotropic medication that is requested by a prisoner between 
three and seven business days prior to the prescription running out will be completed in a 
manner such that there is no interruption or lapse in medication .) only address the 
adequacy of initiation of a medication (i.e. the process of ordering a medication, 
procuring it, and deploying the container to the nurses’ medication cart). PM 12 (Medical 
record will contain documentation of refusals or “no shows. ”) and PM 15 ( Inmates who 
refuse prescribed medication (or no show) will be counseled by a QHCP after three 
consecutive refusals.) only address the adequacy of handling of refusals or no-shows 


90 The patient had one interim visit with a provider on 1/27/19, but this was for a different 
purpose and the mass was not addressed. The patient also had a visit with a provider scheduled 
for 4/18/19, but the patient refused the visit, though there is no indication in the refusal document 
that there was any plan to follow up the mass at this visit. 
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(and, as noted in Part IA, are also counterproductive as currently designed). No PM 
addresses a major component of medication delivery: the administration of deployed 
medications to the patient at each prescribed dosing (unless the patient refuses or no- 
shows). During my review I found examples of failure of nurses to administer 
medications, as illustrated by the following two examples, one involving medical 
medications and one involving mental health medications. 


• Patient 42 suffers from HIV/AIDS, hepatitis C, schizophrenia, and asthma. 
He requires three continuous medications (two of them combined in one 
product; fluticasone/salmeterol) and a fourth intermittent medication 
(“rescue inhaler”) to control his asthma. On 12/30/18 he was transferred to 
the Florence infirmary due to exacerbation of his asthma. Not only was the 
combination medication, which is supposed to be administered twice 
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daily 91 , not transferred with him (an error measured by PM 35, and 
discussed under PM 35 in Part II of my report), but even after the 
medication was procured and deployed on the morning of 1/3/19, nurses 
only provided two of the next 11 doses (See figure below showing 
administration of this medication between 12/29/18 and 1/18/19. The 
arrow (added) shows when the medication first became available, “A” 
indicates the medication was administered, and the circles [added] show 
missed doses). Given the severity of his condition, absence of this 
medication for this long period of time increased the risk that his asthma 
exacerbation would not get better, or would deteriorate, which can result 
in death. 


91 The patient is ordinarily allowed to keep this medication on his person (“KOP”), however, 
upon admission to the infirmary it was converted to a nurse-administered medication. 
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• Patient 39 suffered from schizophrenia with auditory hallucinations, for 

which he was taking an antipsychotic (ziprasidone, Geodon®). On 4/25/19 
he was placed on mental health watch due to delusional thinking. On 
5/6/19 he was transferred from one yard to another yard, both within the 
Eyman complex. The nurse receiving the patient noted that he arrived 
from the first yard without his medications. As shown in the figure below, 
nurses failed to administer his antipsychotic medication on 5/8/19 and 
5/10/19 for no stated reason. (On 5/15/19 nurses did not administer his 
medication, documenting that he refused; “R” in the figure below.) 
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He died from hanging on 5/16/19. The antipsychotic medication he was 
prescribed helps reduce or eliminate the hallucinations and reduce the 
stress caused by the delusions, both symptoms of his schizophrenia. The 
patient was not seen by a mental health professional in the days prior to 
his suicide, so I cannot state with any certainty to what extent his 
hallucinations and/or delusions contributed to his suicide. Further, if they 
did contribute, I cannot state with any certainty to what extent the missing 
of the two doses of medications on 5/8/19 and 5/10/19 contributed to his 
suicide. However, missing the doses of medications does increase the risk. 

Candidate Metric: 

All nurse-administered doses of a prescribed medication are administered, as ordered, or 
there is documentation of a valid reason for non-administration. “As ordered” also means 
the medication was administered at the times of the day ordered or times that would be 
consistent with the pharmacology of the medication (for example rapid-acting or short¬ 
acting insulin is administered shortly before a meal). 

As discussed earlier in this report (Part IA, PM 12), “no-show” should not be considered 
a valid reason for non-administration. As discussed earlier in this report (Part IA, PM 15), 
refusal should be considered valid reason, but only under certain circumstances. 


Mortality Review (MR) 

Issue: 

The MR (including the psychological autopsy) is a critically important element of patient 
safety because it can identify important systematic errors - both those causally related to 
the current death as well as those which are not, but might cause future deaths if left 
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unrecognized - and lead to their remediation. There are three PMs that address the MR 
process: PM 30 ( The initial mortality review of an inmate’s death will be completed 
within 10 working days of death.), PM 31 ( Mortality reviews will identify and refer 
deficiencies to appropriate managers and supervisors, including CQI committee, and 
corrective action will be taken.), and PM 32 {A final independent clinical mortality 
review will be completed by the Health Services Contract Monitoring Bureau for all 
mortalities within 10 business days of receipt of the medical examiner’s findings.). 92 
While these PMs address the need for MR activities to be completed and completed 
within a certain timeframe, they are silent with regard to the adequacy of the MR process. 
There are four aspects of adequacy which are not addressed. 

First, the PMs do not measure whether all significant errors were identified. Second, the 
PMs do not measure whether the root cause of a significant identified error was 
determined. Patient safety science tells us that if the root cause of the error has not been 
identified, a meaningful remedy cannot be found. The following example illustrates both 
errors. It is drawn from just one segment of a very complicated case. 

• Patient 34 was experiencing urinary problems which eventually led to a 

consultation with a urologist on 12/8/16. The urologist discovered blood in 
the patient’s urine, and due to concern for possible cancer, sent a urine test 
for cancer (urine cytology) and asked to see the patient back for follow-up 
in three weeks. The urine test was received at the prison on 12/20/16 and 
was positive for bladder cancer. The following day, 12/21/16, a provider 
made an urgent request for specialty consultation with the urologist. As an 
urgent referral, the visit with the urologist should have taken place by 
1/20/17 (and based on the urologist’s request at the previous visit, the next 
visit with him should have taken place by 1/14/17). Instead, the referral 
was not even sent to the Corizon UM department until 1/30/17. This 
request should have been immediately approved. Instead, on 2/6/17 the 
UM department denied the request, recommending that the providers 
obtain a CT scan. The CT scan was performed on 2/21/17. The results did 
not provide any new information. At this point the patient still needed to 
return to the urologist for further evaluation and treatment of his bladder 
cancer. Instead nothing at all was done. On 7/20/17, during a routine 


92 Also, paragraph 16 of the Stipulation provides “Psychological autopsies shall be provided 
to the monitoring bureau within thirty (30) days of the prisoner’s death and shall be 
finalized by the monitoring bureau within fourteen (14) days of receipt. When a 
toxicology report is required, the psychological autopsy shall be provided to the 
monitoring bureau within thirty (30) days of receipt of the medical examiner’s report. 
Psychological autopsies and mortality reviews shall identify and refer deficiencies to 
appropriate managers and supervisors including the CQI committee. If deficiencies are 
identified, corrective action will be taken.” 
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chronic care visit, he complained of continuing frank blood in his urine, a 
symptom which required immediate referral to a urologist. The mid-level 
provider noted this symptom and wrote in her note that she would refer the 
patient to the urologist. Instead, the provider did nothing for two months 
when, on 9/21/17, she submitted an urgent request for consultation with 
the urologist. As an urgent request, this visit should have occurred by 
10/20/17. Instead it did not occur until 10/25/17. Once seen by the 
urologist on this day, care began for the patient’s bladder cancer. It was 
now deemed invasive and resulted in surgical removal of the bladder and 
prostate. Over the course of the next year he became more ill, and died of 
complications on in May of 2018 93 . Despite the gross errors in care 
described during just this one segment of the patient’s case, the facility 
medical staffs review of his death on 6/7/18 did not identify a single one 
of the critical errors described above, and had not a single 
recommendations for how care could be improved in the future. 

It should be noted that not all errors from all death must be subjected to a root cause 
analysis. However, there should be documentation of a thoughtful process in place to 
inventory the errors and prioritize remedial efforts (including remedial efforts triggered 
by other untoward events). 

Third, while the PMs measure whether there is an intent to remediate identified errors 
(the vendor must cite a corrective action plan in their monthly meeting minutes) the PMs 
do not measure whether the plan was appropriate and sustainable nor whether it was 
actually implemented. 

• For example, Patient 49 died of multi-organ failure in April, 2018. 94 

During an evaluation by a nurse for an emergency a month earlier, the 
nurse failed to appropriately respond to three sets of vital signs which 
were very abnormal (low blood pressure, fast heart rate). Also, apparently, 
the patient had anemia which was not properly evaluated. Corizon medical 
staff noted in his Mortality Review that “health staff reminded to adhere to 
Corizon's "Rules of 100's" when evaluating patients with unstable vitals. 
Adequate work up for anemia should be undertaken in all patients with 
diagnoses of anemia.” Not only does this corrective action plan wholly 
lack any foundation due to the absence of a root cause analysis to 
determine the reason for these errors, even if it had that foundation, the 
plan is devoid of any effectiveness or sustainability: based on patient 


93 I am omitting the exact date to ensure patient confidentiality because dates of death are 
searchable in the public record. 

94 I am omitting the exact date to ensure patient confidentiality because dates of death are 
searchable in the public record. 
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safety science, “reminding” the current staff, for example, is highly 
unlikely to produce any improvement in behavior, and any improvement 
that does occur is highly likely to be short lived (and would certainly not 
carry over to any new employees hired after the “reminder” was issued). 

Fourth, the PMs do not measure whether the implemented remediation was effective. 
Aside from PMs, Corizon did not appear to have any mechanism to check whether 
implemented remediations were effective, based on two findings: Corizon’s standard 
structure (pre-printed form) for conducting its quality improvement (CQI) meetings is 
devoid of a regular part of the meeting to discuss the effectiveness of an remediation 
stemming from deaths or any other untoward event; Corizon’s standard structure for 
tracking quality improvement/remediation following death or other untoward event 
(“Sentinel Event Corrective Action Plan) is also devoid of any mention of measuring the 
effectiveness of remedial actions. During my review of deaths, I encountered problems 
with care related to a death, and would encounter the same problem related to another 
death months later. 

Candidate Metric: 

Following a death, all significant errors are identified. Based on prioritization of all errors 
identified in the organization, root cause analysis is conducted as appropriate, from which 
an effective and sustainable remedial plan is implemented. The remedial plan is 
monitored for effectiveness and appropriate modifications are made to the plan based on 
the monitoring. 
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Exhibit 1 

Complex/PM Pairs With Fewer Than 24 Months of Accumulated Data 


PM# 

PM Description 

Complex 

# Months 
for which 
data is 
available 
as of 
March 
2019 

PM 15 

Inmates who refuse prescribed medication (or no show) will be counseled by a QHCP after 
three consecutive refusals. 

Safford 

17 

PM 25 

A first responder trained in Basic Life Support responds and adequately provides care within 
three minutes of an emergency. 

Douglas 

5 

Perryville 

19 

Safford 

0 

Winslow 

7 

Yuma 

9 

PM 26 

Responses to health care grievances will be completed within 15 working days of receipt (by 
health care staff) of the grievance. 

Douglas 

20 

Winslow 

8 

PM 30 

The initial mortality review of an inmate’s death will be completed within 10 working days of 
death. 

Douglas 

4 

Perryville 

15 

Phoenix 

7 

Safford 

1 

Winslow 

6 

Yuma 

15 

PM 31 

Does the mortality review identify and refer deficiencies to appropriate managers and 
supervisors, including CQI committee, and corrective action plan to be taken? 

Douglas 

5 

Phoenix 

8 

Safford 

5 

Winslow 

11 

Yuma 

20 

PM 32 

A final independent clinical mortality review will be completed by the Health Services 

Contract Monitoring Bureau for all mortalities within 10 business days of receipt of the 
medical examiner’s findings. 

Douglas 

4 

Perryville 

18 

Phoenix 

5 

Safford 

1 

Winslow 

6 

Yuma 

15 

PM 33 

All inmates will receive a health screening by an LPN or RN within one day of arrival at the 
intake facility. 

Eyman 

8 

PM 34 

A physical examination including a history will be completed by a Medical Provider (not a 
dentist) by the end of the second full day of an intake inmate's arrival at the intake facility. 

Eyman 

8 
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PM 40 

Are urgent provider referrals being seen by a Medical Provider within 24 hours of the referral? 

Douglas 

18 

Safford 

17 

PM 62 

All prisoners are screened for tuberculosis upon intake. 

Eyman 

8 

PM 72 

Inmates who refuse prescribed diets for more than 3 consecutive days will receive follow-up 
nutritional counseling by a QHCP. 

Douglas 

20 

Perryville 

12 

Phoenix 

7 

Safford 

15 

Winslow 

22 

PM 75 

A mental health assessment of a prisoner during initial intake shall be completed by mental 
health staff by the end of the second full day after the prisoner’s arrival into ADC. 

Eyman 

6 

PM 76 

If the initial mental health assessment of a prisoner during initial intake is not performed by a 
licensed mental health staff, are the prisoners being seen by a mental health clinician within 
fourteen (14) days of his or her arrival into ADC? 

Eyman 

0 

Perryville 

9 

Phoenix 

2 

Tucson 

3 

PM 83 

MH-3B prisoners who are prescribed psychotropic medications shall be seen a minimum of 
every 180 days by a mental health provider. MH-3B prisoners who are prescribed 
psychotropic medications for psychotic disorders, bipolar disorder, or major depression shall be 
seen by a mental health provider a minimum of every 90 days. 

Phoenix 

15 

PM 84 

MH-3C prisoners shall be seen a minimum of every 180 days by a mental health provider. 

Phoenix 

3 

PM 85 

Are MH-3D prisoners seen by a mental health provider within 30 days of discontinuing 
medications? 

Phoenix 

1 

PM 86 

Are MH-3D prisoners seen a minimum of every 90 days by a mental health clinician for a 
minimum of six months after discontinuing medication? 

Phoenix 

7 

PM 92 

Are MH-3 and above prisoners who are housed in a maximum custody seen by a mental health 
clinician for a 1:1 or group session a minimum of every 30 days? 

Perryville 

21 

PM 93 

Are mental health staff (not to include LPNs) making weekly rounds of all MH-3 and above 
prisoners who are housed in maximum custody? 

Perryville 

21 

Tucson 

17 

PM 95 

Are inmates that are removed from a suicide or mental health watch being removed by a 
licensed mental health staff? Are any prisoners that were discontinued from a suicide or 
mental health watch seen by a mental health provider, mental health clinician, or a psychiatric 
registered nurse between 24 and 72 hours after discontinuation, and between 7 to 10 days, and 
between 21 and 24 days after discontinuation of the watch? 

Douglas 

7 

Safford 

10 

Winslow 

21 

PM 97 

Are mental health providers treating a prisoner via telepsychiatry, provided (in advance of the 
telepsychiatry session), the prisoner’s intake assessment, most recent mental health treatment 
plan, laboratory reports (if applicable), physician orders, problem list, and progress notes from 
the prisoner’s two most recent contacts with a mental health provider? 

Douglas 

22 

Safford 

14 

Winslow 

14 
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PM 

Prisoners on the routine dental care list will not be removed from the list if they are seen for 
urgent care or pain appointments that do not resolve their routine care issues or needs. 

Douglas 

16 

100 


Florence 

17 



Lewis 

22 



Perryville 

22 



Phoenix 

12 



Safford 

9 



Tucson 

23 



Winslow 

5 



Yuma 

12 
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